Select proper KafkaUser authentication type?

Question

Maybe I miss something, if so forgive my ignorance.

Here what we have:

1. We use TLS authentication listeners in Kafka cluster (this can be changed, we can add new type of listeners).
2. When connect to Kafka topic from Java code I use SSL certificate generated for the Kafka user.

If I decide to avoid using SSL certificate, because of 2 reasons:

1. I will connect to Kafka topic only from trusted OpenShift cluster PODs
2. To avoid updating on producer/consumer side re-generareated yearly user's SSL certificate (because Kafka generates user certificate 1 year valid period)

Would be the SCRAM-SHA-512 authentication type for KafkaUser a better (and the only ?) choice for the two reasons above? Or SCRAM-SHA-512 also requires SSL certificates?

Another approach I saw was no authentication, but I am not sure how can ACL be used for such users? How I pass to server information which user is connecting. Is it possible to use both ACL and not authenticated by SSL certificate or by password Kafka user?

[UPD] Environment is built on Strimzi (Apache Kafka cluster in OpenShift)

Answer 1

Using SCRAM-SHA-512 does not require TLS. So you can just disable the TLS encryption in the Kafka custom resource (.spec.kafka.listeners -> set tls: false), enable he SCRAM-SHA-512 authentication (same place, in the authentication section). And then you just use the KafkaUser to create the user and get the password.

In general, TLS encryption is normally always recommended. But the SCRAM-SHA mechanisms do not send the password over the network directly, so using it without encryption should not leak the password. At the end, it is up to you to decide.

Also, just as a sidenote - the certificates are for 1 year by default. You can change it in the Kafka CR.