Why Karafka Server Throw error query watermark offset?

Question

I'm using this gem for my Ruby on Rails Apps to produce and consume message from kafka server. https://karafka.io/docs/

and this is my karafka.rb file

# frozen_string_literal: true

class KarafkaApp < Karafka::App
  setup do |config|
    config.kafka = {"bootstrap.servers": ENV["KAFKA_BOOTSTRAP_SERVERS"]}
    config.client_id = "store"
    config.consumer_persistence = !Rails.env.development?
  end

  Karafka.monitor.subscribe(Karafka::Instrumentation::LoggerListener.new)
  
  Karafka.producer.monitor.subscribe(
    WaterDrop::Instrumentation::LoggerListener.new(Karafka.logger)
  )

  Karafka.monitor.subscribe "error.occurred" do |event|
    type = event[:type]
    error = event[:error]
    details = (error.backtrace || []).join("\n")

    puts "Oh no! An error: #{error} of type: #{type} occurred!"
    puts details
    puts "=" * 100
    NewRelic::Agent.notice_error(error)
  end

  routes.draw do
    topic "payment-order" do
      consumer PaymentOrderConsumer
    end

    topic "payment-method-config" do
      consumer PaymentMethodConsumer
    end

    topic "order-refund" do
      consumer OrderRefundConsumer
    end
  end
end

Karafka::Web.enable!

After quite sometimes, i got this error Error querying watermark offsets for partition 0 of karafka_consumers_states - Local: Unknown partition (unknown_partition) The consumer still consuming new message without problem, but i got error above periodically.

I'm not found another error on google.

I've tried install bash and librdkafka on server. Only using 1 kafka bootstrap server.

This is the topic already present on the kafka server.

Answer 1

I'm the Karafka author.

You did not follow the instructions thoughtfully. The web-ui requires a bootstrap: https://karafka.io/docs/Web-UI-Getting-Started/

Point 3:

Run the following command to install the karafka-web in your project:

bundle exec karafka-web install

Without that, Karafka Web-UI cannot get the expected data to operate.

You can either run the install or you can disable the web-ui. The choice is yours.

Karafka Web-UI provides a status page that can help you identify and mitigate problems that would cause the Web UI to malfunction or misbehave. If you see the 404 page or have issues with Karafka Web UI, this page is worth visiting.