Can I use Prometheus metric HELP description in alert definition

266 views Asked by At

Given I can create this example Prometheus metric:

HELP some_metric This is the metric description
TYPE some_metric counter
some_metric{job="foo",instance="a",some_label="value"} 5

I'd like to be able to use the metric description from the HELP annotation in the Prometheus alert definition like this:

annotations:
  description: This is HELP {{ $meta.HELP }}
  summary: And this is TYPE {{ $meta.TYPE }}

Note that I am currently interested only in the HELP, but providing TYPE too seems like a logical thing to do.

I know I can use $value, $labels, and $externalLabels variables, but I can see no help/meta or anything.

1

There are 1 answers

1
kholisrag On

You can, but you need to do it manually in prometheus rules, it can't be automatic from expr

like

alert: GitlabRunnerFatalErrorCaught
expr: sum
  by(instance, service, level) (rate(gitlab_runner_errors_total{job="gitlab-runner-monitoring",level="fatal"}[5m]))
  * 60 * 5 > 0
for: 5m
labels:
  severity: P2
annotations:
  description: Gitlab Runner {{ $labels.service }} - {{ $labels.instance }} have an
    {{ $labels.level }} level error
  summary: Gitlab Runner systemd have an error

you could add the description/summary from :

  • annotations.description
  • annotations.summary

Hopefully answer your question