I want to achieve something within the current kube-prometheus-stack we have. I would like to have a dynamic severity based on a map of severity/job.
What I have now within the kube-prometheus-stack:
config:
templateFiles:
determineSeverity.tmpl: |-
{{/* Determine severity based on job */}}
{{/* Argument list of $jobAlertLevelMap and $job */}}
{{/* Returns $alertLevel, default= Medium */}}
{{- define "monitoring.determineSeverity" }}
{{- $job := . -}}
{{- $alertLevel := "Medium" }}
{{- range $key, $value := $.Values.alertLevelPerJob }}
{{- if has $job $value }}
{{- $alertLevel = $key }}
{{- end }}
{{- end }}
{{- $alertLevel | title | quote }}
{{- end }}
additionalPrometheusRulesMap:
rule-name:
groups:
- name: General
rules:
- alert: Watchdog
expr: vector(1)
labels:
severity: '{{ template "monitoring.determineSeverity" "Watchdog"}}'
alertLevelPerJob:
critical:
- gitlab
medium:
- confluence
- Watchdog
I tried several things but unfortunately it's not working, someone any tips?