Several Event Types in the Same Topic With Kafka, Quarkus and Avro

1.2k views Asked by At

I am trying to read an event from a topic using Quarkus. The topic could contain different types of events. All the events use the AVRO format, so I have a schema registry where I can read the releted schema of an event. I use avro-maven-plugin to compile the schema into a Java class.

Suppose we have two types of events with the following schemas:

Event1

{
   "field1": "string"
}

Event2

{
   "field2": "string"
}

In my application I am interested only on one of them.

 public class Consumer {

    @Incoming("test-in")
    public CompletionStage<Void> read(
        IncomingKafkaRecord<String, Event1> data) {
        System.out.println(data.getKey());
        System.out.println(data.getPayload());
        return data.ack();
    }
}

This code print all the events not only the evnts of type Event1 as I expected.

When I try to get data from the event data.getField1() I get a CastException.

java.lang.ClassCastException: class org.apache.avro.generic.GenericData$Record cannot be cast to class com.test.Event1 (org.apache.avro.generic.GenericData$Record is in unnamed module of loader io.quarkus.bootstrap.classloading.QuarkusClassLoader @1aa7ecca; com.test.Event1 is in unnamed module of loader io.quarkus.bootstrap.classloading.QuarkusClassLoader @1144a55a)

There is a way to read only some event types from a topic with several event types usign AVRO?

1

There are 1 answers

0
theShadow89 On

A possible solution (little bit verbouse) is using SpecificRecord as payload type and set specific.avro.reader=true on application configuration file for incoming events.

 public class Consumer {

    @Incoming("test-in")
    public CompletionStage<Void> read(
        IncomingKafkaRecord<String, SpecificRecord> data) {
            String schemaFullName = data.getPayload().getSchema().getFullName();
            if (schemaFullName.equals(Event1.class.getName())) {
                System.out.println(((Event1) data.getPayload()).getField1());
            }
        return data.ack();
    }
}

After selecting the event you can cast the SpecificRecord to the correct compiled Avro event class.