exposing kafka topic LAGs using kafka jmx exporter

Question

I am trying to expose the Prometheus metric that shows LAG value in each Kafka topic. I use kafka_exporter for exporting Kafka metrics.

I added this pattern in prom-jmx-agent.config.yaml

lowercaseOutputName: true
rules:
- pattern: kafka.consumer<type=consumer-fetch-manager-metrics, client-id=(.+), topic=(.+), partition=(.+)><>(records-lag)
  name: kafka_consumer_fetch_records_lag
  labels:
    clientId: "$1"
    topic: "$2"
    partition: "$3"
    help: "Kafka Consumer Fetch Records Lag"
    type: GAUGE

I create some lags to be able to see in the Prometheus

Describe consumer group

I expect to see metric named kafka_consumer_fetch_records_lag but it does not appear in the metric list.

Answer 1

The above JMX exporter config must be added to JVM-based consumer clients. The brokers don't have kafka_consumer_fetch_records_lag metric

If you want to find lag of all clients (including non JVM ones),

• https://github.com/seglo/kafka-lag-exporter
• https://github.com/linkedin/Burrow
• https://github.com/zalando-incubator/remora