Error opening zip file or JAR manifest missing : /etc/jmx_exporter/jmx_prometheus_javaagent.jar

Question

I'm trying to add JMX exporter on my kafka-zookeeper. For this I've followed these steps:

• copied jmx_prometheus_javaagent.jar to the current directory
• mounted volume with /jmx_prometheus_javaagent.jar:/etc/jmx_exporter
• and added the argument like this: KAFKA_OPTS: -javaagent:/etc/jmx_exporter/jmx_prometheus_javaagent.jar=7071:/etc/jmx_exporter/kafka_config.yml

I'm downloading jmx_prometheus_javaagent-0.20.0.jar from here https://github.com/prometheus/jmx_exporter/releases/tag/0.20.0. And remaning it to jmx_prometheus_javaagent.jar.

Here is the complete docker-compose.yml file:

version: '3.6'
name: "kafka-zookeeper"
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:7.3.0
    container_name: zookeeper
    ports:
      - "2181:2181"
      - "9997:9997" # JMX port for Zookeeper
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
      KAFKA_JMX_PORT: 9997          # JMX port for Zookeeper
      KAFKA_JMX_HOSTNAME: localhost # JMX port for Zookeeper

  kafka:
    image: confluentinc/cp-kafka:7.3.0
    container_name: kafka
    ports:
      - "9092:9092"
      - "9998:9998" # JMX port for Kafka
      - "7071:7071" # JMX Exporter port
    volumes:
      - /jmx_prometheus_javaagent.jar:/etc/jmx_exporter  # JMX exporter mount
    depends_on:
      - zookeeper
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_INTERNAL:PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092,PLAINTEXT_INTERNAL://kafka:29092
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
      KAFKA_JMX_PORT: 9998          # JMX port for Kafka
      KAFKA_JMX_HOSTNAME: localhost # JMX host for Kafka
      KAFKA_OPTS: -javaagent:/etc/jmx_exporter/jmx_prometheus_javaagent.jar=7071:/etc/jmx_exporter/kafka_config.yml # JMX exporter config

Right now, I'm stuck and don't know what to do with it? I surely can't use other solutions, as everything is self hosted, on top of that having multiple dashboard to monitor stuffs will be really absurd.

Complete log at pastebin: https://pastebin.com/NWyYuxs6

Answer 1

Ended up using helm chart for kafka, with the following details:

image:

image:
  registry: docker.io
  repository: bitnami/kafka
  tag: 3.5.0-debian-11-r4
  digest: ""

And it has builtin JMX support to it. All I had to do is, enable it:

  jmx:
    ## @param metrics.jmx.enabled Whether or not to expose JMX metrics to Prometheus
    ##
    enabled: true

Answer 2

don't know what to do with it

You need to run Prometheus server and setup a scrape_config to point at kafka:7071/metrics

Otherwise, you can open localhost:7071/metrics to see the metrics that are being gathered by the exporter

surely can't use other solutions

You could. Such as MiniKube + Strimzi, rather than Docker Compose, which comes with JMX exporter already