How does a python kafka producer work when some of brokers are not available?

547 views Asked by At

I have set up a 3 node kafka cluster and used python as producer like this:

kafka_addr = "n0.xxx.com:9092,n1.xxx.com:9092,n2.xxx.com:9092"
producer = KafkaProducer(bootstrap_servers=kafka_addr)

When "n0" and "n1" are available but "n2" is not available(broker breakdown or network error),the producer couldn't work normally by sending to "n0n1" but throw an error :

getaddrinfo failed for n2.xxx.com:9092, exception was [Errno 8] nodename nor servname provided, or not known. Is your advertised.host.name correct and resolvable?
1

There are 1 answers

0
HenryLulu On

I try to find an available broker and send to it:

kafka_addr = ["n0.xxx.com:9092","n1.xxx.com:9092",n2.xxx.com:9092"]
producer = None
for broker in kafka_addr:
    try:
        producer = KafkaProducer(bootstrap_servers=broker)
        break
    except:
        pass
if producer is not None:
    #do something