Dynamic variables in Prometheus alert rule

64 views Asked by At

I have a rule that monitors the memory usage of a container compared to the limit set to it:

- alert: high_memory_usage_resource_limits
  expr: >
    (
      sum by (container, pod, namespace) (container_memory_usage_bytes)
    ) 
    / 
    (
      sum by (container, pod, namespace) (kube_pod_container_resource_limits{resource="memory"})
    ) * 100 > 90
  for: 1m
  labels:
    severity: warning
  annotations:
    summary: "High memory usage compared to resource limits on {{$labels.pod}}"
    description: "Container {{$labels.container}} in pod {{$labels.pod}} is consuming more memory than its resource limit ({{$value}}% of the limit)"

I have 2 questions:

  1. Is it possible to configure the threshold (>90) as a variable and then re-use it in the message as well?
  2. Can my message contain the limit fetched from the query? For example, if my Pod is limited to use 1Gi of memory, I would like to print that value in the message.
0

There are 0 answers