Confluent Platform: version-alignment between Confluent Platform and Apache Kafka releases

163 views Asked by At
OS: ubuntu 18.x
Confluent Schema Registry:  community edition version 5.5.1
Kafka:  AWS MKS (Apache Kafka 2.2.1)

I am circling around several issues in relation to expunging topic logs that reference stale schema IDs. That specific issue was previously reported. Here, I elaborate on related matters.

I decided to delete all logs for my topic -- call it TOP. Following several articles (including one from Confluent docs), I tried setting the retention time for my topic to 1s. Eg:

$ ./kafka-topics --bootstrap-server "b-3.amazonaws.com:9092,b-2.amazonaws.com:9092,b-1.amazonaws.com:9092" --alter --config retention.ms=1000 --topic TOP
Option combination "[bootstrap-server],[config]" can't be used with option "[alter]"
...

As the output shows here, that command fails and suggests an inappropriate combination of command-line options. I then tried something similar with kafka-configs command:

$ ./kafka-configs --bootstrap-server "b-3.amazonaws.com:9092,b-2.amazonaws.com:9092,b-1.amazonaws.com:9092" --entity-type topics --entity-name TOP --alter --add-config retention.ms=1000
Error while executing config command with args '--bootstrap-server b-3.amazonaws.com:9092,b-2.amazonaws.com:9092,b-1.amazonaws.com:9092 --entity-type topics --entity-name call --alter --add-config retention.ms=1000'
java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.UnsupportedVersionException: The broker does not support INCREMENTAL_ALTER_CONFIGS
    at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
    at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
    at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:104)
    at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:272)
    at kafka.admin.ConfigCommand$.alterConfig(ConfigCommand.scala:327)
    at kafka.admin.ConfigCommand$.processCommand(ConfigCommand.scala:298)
    at kafka.admin.ConfigCommand$.main(ConfigCommand.scala:91)
    at kafka.admin.ConfigCommand.main(ConfigCommand.scala)
Caused by: org.apache.kafka.common.errors.UnsupportedVersionException: The broker does not support INCREMENTAL_ALTER_CONFIGS

So, I am suspicious there is a disconnect between the Confluent docs I am referencing and the version of Kafka I am using through AWS MKS. In this circumstance, what is the recommended practice for setting the topic-specific configuration property?

More generally, can anyone clarify the version alignment between Apache Kafka releases and Confluent Platform?

0

There are 0 answers