I have a rule that monitors the memory usage of a container compared to the limit set to it:
- alert: high_memory_usage_resource_limits
expr: >
(
sum by (container, pod, namespace) (container_memory_usage_bytes)
)
/
(
sum by (container, pod, namespace) (kube_pod_container_resource_limits{resource="memory"})
) * 100 > 90
for: 1m
labels:
severity: warning
annotations:
summary: "High memory usage compared to resource limits on {{$labels.pod}}"
description: "Container {{$labels.container}} in pod {{$labels.pod}} is consuming more memory than its resource limit ({{$value}}% of the limit)"
I have 2 questions:
- Is it possible to configure the threshold (
>90) as a variable and then re-use it in the message as well? - Can my message contain the limit fetched from the query? For example, if my Pod is limited to use 1Gi of memory, I would like to print that value in the message.