I've been trying to add a SASL Authentication to my Kafka Brokers using SASL_PLAINTEXT SCRAM-SHA-256 for a while, but without any success. I keep getting the following error on Kafka's logfile.
ERROR [Controller id=0, targetBrokerId=0] Connection to node 0 failed authentication due to: Authentication failed due to invalid credentials with SASL mechanism SCRAM-SHA-256 (org.apache.kafka.clients.NetworkClient).
But I've been following Kafka docs on SCRAM Config to the letter and not getting anywhere near of successfully achieving this to work.
I registred an admin user on Zookeeper using kafka-configs.sh like below:
bin/kafka-configs.sh --zookeeper localhost:2181 --alter --add-config 'SCRAM-SHA-256=[password=admin-secret],SCRAM-SHA-512=[password=admin-secret]' --entity-type users --entity-name admin
Here are part of my server.properties where I configure SCRAM-SHA-256.
broker.id=50
sasl.enabled.mechanisms=SCRAM-SHA-256
sasl.mechanism.inter.broker.protocol=SCRAM-SHA-256
security.inter.broker.protocol=SASL_PLAINTEXT
listeners=SASL_PLAINTEXT://172.16.3.21:9092
advertised.listeners=SASL_PLAINTEXT://172.16.3.21:9092
listener.name.sasl_plaintext.scram-sha-256.sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
username="admin" \
password="admin-secret";
Generating CA on one node and copying it to other node manually worked for me
STEP 1- RUN this on each node
keytool -keystore server.keystore.jks -alias kafka -dname "cn=kafka, ou=it, o=stackoverflow, l=alpha, st=beta, c=IN" -storepass test123 -validity 365 -keyalg RSA -genkey -ext SAN=DNS:kafka-host1,DNS:kafka-host2,DNS:kafka-host3,DNS:localhost,DNS:kafka
STEP 2 - verify the cert
keytool -list -v -keystore server.keystore.jks
STEP 3 - generate this once on single node and copy is to other nodes.
openssl req -new -x509 -keyout ca-key -out ca-cert -days 365 -subj "/C=IN/ST=beta/O=stackoverflow/OU=it/L=alpha/CN=kafka"
REMAINING STEPS ARE BELOW NEED TO RUN ON EACH NODES
keytool -keystore server.truststore.jks -alias CARoot -import -file ca-cert -storepass test123 -noprompt
keytool -keystore client.truststore.jks -alias CARoot -import -file ca-cert -storepass test123 -noprompt
keytool -keystore server.keystore.jks -alias kafka -certreq -file cert-file -storepass test123
openssl x509 -req -CA ca-cert -CAkey ca-key -in cert-file -out cert-signed -days 3650 -CAcreateserial -passin pass:test123
keytool -keystore server.keystore.jks -alias CARoot -import -file ca-cert -storepass test123 -noprompt
keytool -keystore server.keystore.jks -alias kafka -import -file cert-signed -storepass test123 -noprompt