How to fix KafkaTimeoutError when connecting to Red Panda using kafka-python

Question

I get the following error when trying to connect a kafka-python producer to my Red Panda database.

KafkaTimeoutError: KafkaTimeoutError: Failed to update metadata after 60.0 secs.

I have a development environment running on WSL with the Red Pandas container on docker desktop. Red Panda is allocated 1GB of RAM so I don't believe it is a memory issue.

I used the following to connect to my Red Panda broker:

producer = KafkaProducer(
    bootstrap_servers = '192.168.2.28:19092', 
    security_protocol = 'SSL',
    api_version=(0, 10, 1)
)

I had to add both the security_protocol and the api_version in order for the producer to connect. I can assume it's connected as I am not getting any connection errors when that code is run.

I then try to send test data using the following and I receive the error:

producer.send('test', b'Hello, World!')

I have also tried not assigning the text as bytes and I get the same error.

Does anyone know how to fix this error or have experience connecting producers to Red Panda, I am also open to using confluent-kafka.

Answer 1

You don’t need SSL if you didn’t enable SSL in Redpanda. If you enabled SSL, you will also need to configure the cert or keystore in your producer. I don’t think the API version is relevant though. Hope that helps.