PromQL get both delta result and current value?

Question

I have the following query:

delta(elasticsearch_cluster_health_number_of_nodes[1m]) < 0

which will return:

{cluster="elasticsearch", container="exporter", endpoint="http", environment="production", instance="x.x.x.x", job="elasticsearch-exporter", namespace="elasticsearch", pod="elasticsearch-exporter-76b7b4b85c-ggds4", prometheus="monitoring/victoria-metrics-vm", service="elasticsearch-exporter"} -1

Now, I want to have access to the current number of pods as well. So I tried this:

union(
    delta(elasticsearch_cluster_health_number_of_nodes[1m]) < 0, 
    alias(sum(elasticsearch_cluster_health_number_of_nodes[1m]) by(cluster), "current_state"),
)

It will return two results (delta and sum) as expected.

The question is - is there a better way of doing this? Also, how can I access the value of aliased query in my alert template?

Answer 1

The question is - is there a better way of doing this? Also, how can I access the value of aliased query in my alert template?

In alerts you can do extra queries right in annotations:

  - alert: Conns
    expr: vm_tcplistener_conns > 0
    for: 3m
    annotations:
      labels: "Available labels: {{ $labels }}"
      summary: Too high connection number for {{ $labels.instance }}
        {{ with printf "sum(vm_tcplistener_conns{instance=%q})" .Labels.instance | query }}
          {{ . | first | value }}
        {{ end }}
      description: "It is {{ $value }} connections for {{$labels.instance}}"

See https://docs.victoriametrics.com/vmalert.html#template-functions