Abinitio - how to read and parse Avro data in Kafka?

83 views Asked by At

I am new to Abinitio and have a requirement to parse confluent Kafka AVRO messages using Confluent schema registry. Am using Kafka subscriber component. The output port (out) of Kafka Subsriber is below (default) :

include "/~ab_components/Datasets/Kafka/kafka_types.dml";
metadata type = kafka_subscribe_10_t;

The output/record looks like below in the watcher:

Record 1:0:

[record
  partition                 2
  offset                    91958
  timestamp                 1699422363749
  timestampType             0
  headers                   [vector]
  key                       0000000065403246374444...
  value                     0000001631234567890112233445566...
  timestamp_as_dml_datetime 2021-02-08 05:46:03.749]

Record 2:0:

[...
]

How do I further parse/read/translate above AVRO messages from Kafka ? What are the subsequent components required in the pipeline - (Read Avro) ? Please provide a working sample flow with details on each components like their parameters, ports, package, dml etc.

Thanks!

0

There are 0 answers