I am publishing messages via golang using segmentio-go. And I want to read those message using python for which I tried kafka-python, pykafka.
In all these libraries, I cannot receive the message, I am able to read the messages when reading it from golang by creating a consumer at golang side. And also when I am publishing messages via python, I am able to read it from the python consumer as well as from golang consumer, but those topics in which messages are published via golang, consumer in python side gets completely hanged/stuck, code doesn't crash but also don't respond.
In the python consumer, I am able to get broker and its topics, It is only for those topics in which messages are published via golang, python consumer gets hanged.
Code for Python-consumer:
consumer = KafkaConsumer(<topic name>, bootstrap_servers=[<ip:port>], auto_offset_reset='earliest', group_id=None, max_partition_fetch_bytes=104857600)
print(consumer.topics()) # Able to get all topic names
for message in consumer:
print(message.value)
Code for Golang Producer
l := log.New(os.Stdout, "kafka framer: ", 0)
w := kafka.NewWriter(kafka.WriterConfig{
Brokers: []string{broker_address},
Topic: topic_name,
Logger: l,
})
err := w.WriteMessages(ctx, kafka.Message{
Key: []byte(strconv.Itoa(0)),
Value: []byte(data),
})
if err != nil {
panic("could not write message " + err.Error())
}
Please help.