Using kube-prometheus-stack how to find metrics in other namespaces?

Question

Using 52.1.0 of the kube-prometheus-stack what is the correct way to get metrics running in other namespaces.

I have a windows-exporter running on port 5000 named metrics under a different namespace and I've followed many blogs / guides etc but none seem to have done the trick of getting the metrics collected.

Are there some docs on how to do this? Or a way to debug why its not being picked up?

I have this in my values.yaml but it doesn't seem to be enough to find the endpoint which I'm assuming is because my target is running in a sidecar container in a different namespace to where my prometheus stack is running.

  prometheus:
    enabled: true

    additionalServiceMonitors:
    - name: "prometheus-windows-pod-exporter-monitor"
      selector:
        matchLabels:
          appType: web
      endpoints:
        - port: "metrics"

Answer 1

From this document:

In order to monitor additional namespaces, the Prometheus server requires the appropriate Role and RoleBinding to be able to discover targets from that namespace. By default the Prometheus server is limited to the three namespaces it requires: default, kube-system and the namespace you configure the stack to run in via $.values.namespace.

You can try adding namespaces in ServiceMonitor.

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: k8s-apps-http
  labels:
    k8s-apps: http
spec:
  jobLabel: k8s-app
  selector:
    matchExpressions:
    - {key: k8s-app, operator: Exists}
  namespaceSelector:
    matchNames:
    - kube-system
    - monitoring
    -<add you namespace>
  endpoints:
  - port: http-metrics
    interval: 15s

For more information you can follow this document.