Trouble with Apache Kafka to Allow External Connections

Question

I'm just having a difficult time with Kafka right now, but I feel like I'm close.

I have two VMs on FreeNAS running locally. Both Running Ubuntu 18.04 LTS.

VM Graylog: 192.168.1.25. Running Graylog Server. Working well retrieving rsyslogs and apache from itself.

VM Kafka: 192.168.1.16. Running Kafka.

My goal is to have VM Graylog pull logs from VM Kafka, via a Graylog Kafka UDP input. The secondary goal is to replicate this, except tha the Kafka instance will sit on my VPS server feeding apache logs from a website. Of course, I want to test this in a dev environment first.

I am able to have my VM Kafka server successfully listen through this line of code:

/opt/kafka_2.13-2.6.0/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic rsyslog_kafka --from-beginning

This is my 60-kafka.conf file:

module(load="omkafka")

template(name="json"
         type="list"
         option.json="on") {
           constant(value="{")
             constant(value="\"@timestamp\":\"")     property(name="timereported" dateFormat="rfc33$
             constant(value="\",\"@version\":\"1")
             constant(value="\",\"message\":\"")     property(name="msg")
             constant(value="\",\"host\":\"")        property(name="hostname")
             constant(value="\",\"severity\":\"")    property(name="syslogseverity-text")
             constant(value="\",\"facility\":\"")    property(name="syslogfacility-text")
             constant(value="\",\"programname\":\"") property(name="programname")
             constant(value="\",\"procid\":\"")      property(name="procid")
           constant(value="\"}\n")
         }

action(
  broker=["192.168.1.16:9092"]
  type="omkafka"
  topic="rsyslog_kafka"
  template="json"
)

I'm using the default server.properties file which doesn't contain any listeners, just the defaults. I do understand I need to set the listeners and advertised.listeners. I've attempted the following settings to no avail:

Attempt 1: listeners = PLAINTEXT://localhost:9092 advertised.listeners=PLAINTEXT://192.168.1.16:9092

Attempt 2: listeners = PLAINTEXT://127.0.0.1:9092 advertised.listeners=PLAINTEXT://192.168.1.16:9092

This after reloading both Kafka and Rsyslog and confirming their statuses are active.

Example errors when attempting to read messages. Bunch of these [2020-08-20 00:52:42,248] WARN [Consumer clientId=consumer-console-consumer-70205-1, groupId=console-consumer-70205] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)

Followed by an infinite amount of these: [2020-08-20 00:48:50,598] WARN [Consumer clientId=consumer-console-consumer-11975-1, groupId=console-consumer-11975] Error while fetching metadata with correlation id 254 : {rsyslog_kafka=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)

I feel like I'm close. Perhaps there is something I'm just understanding. I've read lots of similar articles where they say just replace the IP addresses with your server. I feel like I've done that, with no success.

Answer 1

You need to set listeners to PLAINTEXT://0.0.0.0:9092 in order to bind externally.

The advertised listener ought to be set to an address that your consumers will be able to use to discover the cluster

Note: Docker Compose might be easier than VMs