I know Kafka Consumer and broker exchange Heartbeat messages during idling(application messages not sent) periods (governed by heartbeat.interval.ms in librdkafka).
Is similar heartbeat messaging scheme used between Kafka Producer and Broker ?
If yes,
- what are the contents of such heartbeat TCP packet? or is it a TCP packet without payload ?
- in librdkafka what is the configuration that controls heartbeat interval and maximum allowed heartbeat miss count ?
If NO,
- some firewall rules break idling TCP connections. How can such a situation be handled ? Is there a mechanism to send a message from producer without it being written on topic ?
EDIT: As per related question Prevent Kafka broker from closing idle connection, Author of librdkafka says "The Kafka protocol does not really have a cheap or side-effect free way of "heartbeating" (not to be confused with the HeartbeatRequest) an idle connection." in 2017.
But librdkafka has property socket.keepalive.enable (Enable TCP keep-alives (SO_KEEPALIVE) on broker sockets). Which seems to be added before 2017 (in 2015) as per https://github.com/confluentinc/librdkafka/issues/283