I am new to Abinitio and have a requirement to parse confluent Kafka AVRO messages using Confluent schema registry. Am using Kafka subscriber component. The output port (out) of Kafka Subsriber is below (default) :
include "/~ab_components/Datasets/Kafka/kafka_types.dml";
metadata type = kafka_subscribe_10_t;
The output/record looks like below in the watcher:
Record 1:0:
[record
partition 2
offset 91958
timestamp 1699422363749
timestampType 0
headers [vector]
key 0000000065403246374444...
value 0000001631234567890112233445566...
timestamp_as_dml_datetime 2021-02-08 05:46:03.749]
Record 2:0:
[...
]
How do I further parse/read/translate above AVRO messages from Kafka ? What are the subsequent components required in the pipeline - (Read Avro) ? Please provide a working sample flow with details on each components like their parameters, ports, package, dml etc.
Thanks!