Kafka - Spring : kafka consumer read a message from topic based on offset

Question

Is there a way to consume a message from Kafka topic based on offset. I mean I have a offset id that I previously published in a topic. Now I need get a message from topic based on offset Id which I'm passing.

Answer 1

Using Java Kafka Consumer Library, However you have to know the partition number also.

KafkaConsumer<String, String> consumer = new KafkaConsumer<String, String (properties);
long desiredOffset = 10000;
TopicPartition partition = new TopicPartition("some-topic", 0);
consumer.assign(Arrays.asList(partition));
consumer.seek(partition, desiredOffset);
bool found= false;
while(found != true){
    ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(100));
    for(ConsumerRecord<String,String> record: records){
        if(record.offset() == desiredOffset){
            System.out.println(record)
            found= true;
            break;
        }
    }
}

consumer.close();

Things to consider, the record with your desired offset can be deleted depending of the clean up policy configuration in your Kafka Topic. Remember Kafka is a stream platform. Read the message by offset only if you are debugging.

Answer 2

Simply use Kafka consumer with required parameters like

• bootstrap-server : (comma separated server names : port no.)
• topic : (topic name)
• partition : (partition number)
• offset : (offset value)
• max-messages : (No. of maximum messages to consume)

sh kafka-console-consumer.sh --bootstrap-server server1:9092,server2:9092,server3:9092 --topic test_topic --partition 0 --offset 43212345 --max-messages 1