How can I stop Dapr Kafka Component from connecting to closed consumer groups?

Question

I have the following component in k8s:

apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: kafka-pubsub
  namespace: default
spec:
  type: pubsub.kafka
  version: v1
  metadata:
      # Kafka broker connection setting
    - name: brokers
      value: "kafka-0.kafka-headless.default.svc.cluster.local:9092, 
kafka.default.svc.cluster.local:9092"
    - name: authRequired
      value: "false"
    - name: saslUsername
      value: ""
    - name: saslPassword
      value: ""
scopes:
  - signalrconnectorinterface
  - cardhandler
  - eventtrigger

I have a number of microservices which all subscribe properly, however the ones which subscribe to two topics (and more presumably) get the following error message upon subscription in the daprd logs:

time="2023-08-04T13:41:54.24563974Z" level=info msg="initializing Dapr workflow component" app_id=cardhandler component="dapr (workflow.dapr/v1)" instance=cardhandler-deployment-ff8d86465-xt85c scope=dapr.contrib type=log ver=1.11.2
time="2023-08-04T13:41:54.263847422Z" level=info msg="app is subscribed to the following topics: [send-card receive-command-message] through pubsub=kafka-pubsub" app_id=cardhandler instance=cardhandler-deployment-ff8d86465-xt85c scope=dapr.runtime type=log ver=1.11.2
time="2023-08-04T13:41:54.267674781Z" level=info msg="placement tables updated, version: 0" app_id=cardhandler instance=cardhandler-deployment-ff8d86465-xt85c scope=dapr.runtime.actor.internal.placement type=log ver=1.11.2
time="2023-08-04T13:41:55.200932919Z" level=error msg="Error consuming [receive-command-message]. Retrying...: kafka: tried to use a consumer group that was closed" app_id=cardhandler component="kafka-pubsub (pubsub.kafka/v1)" instance=cardhandler-deployment-ff8d86465-xt85c scope=dapr.contrib type=log ver=1.11.2
time="2023-08-04T13:41:58.812244394Z" level=info msg="dapr initialized. Status: Running. Init Elapsed 4843ms" app_id=cardhandler instance=cardhandler-deployment-ff8d86465-xt85c scope=dapr.runtime type=log ver=1.11.2

It notices there are two subscriptions, tries to allocate them then fails. Any advice?

Answer 1

As per OneCricketeer, there is actually no issue here, this is intended behaviour. For some reason this just started happening when I was using a new helm install of Apache Kafka.

Ended up being a codebase issue so reset and all fine :)