How to convert a working Prometheus query into a Prometheus adapter rule

507 views Asked by At

Here is a query that is working well in Prometheus.

rate(starlette_requests_total{method="POST", path_template="/predict"}[1m])

prometheus-image

Prometheus stack is installed via a helm chart:

kubectl create namespace monitoring && \
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts && \
helm repo update && \
helm install -n monitoring prometheus-stack prometheus-community/kube-prometheus-stack

I tried to make the rule as follows:

helm install -n monitoring myprom-adapter prometheus-community/prometheus-adapter -f values.yaml

where values.yaml contains:

rules:
  custom:
  - seriesQuery: 'starlette_requests_total'
  resources:
    overrides:
      kubernetes_namespace: {resource: "namespace"}
      kubernetes_pod_name: {resource: "pod"}
    name:
      matches: "starlette_requests_total"
      as: "prediction_requests_rate_1m"
    metricsQuery: rate(starlette_requests_total{method="POST", path_template="/predict"}[1m])

The metric doesn't show up in the custom metric:

# kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1                                                                                                                      
{"kind":"APIResourceList","apiVersion":"v1","groupVersion":"custom.metrics.k8s.io/v1beta1","resources":[]}

I'm running these on a Minikube cluster (v1.26.0) with these helm chart versions: prometheus-adapter-3.3.1 and kube-prometheus-stack-37.2.0

What would a corresponding Prometheus Adapter Rule look like to get this exposed in the custom-metrics API for HPA to use?

0

There are 0 answers