I have a kafka implementation without any schema validation.
The Kafka producer is basically publishing a Person Message to Kafka using Avro Serializer.
I'm using the avro-maven-plugin to generate the Person pojo class from a avro schema.
Here is the avro schema i have design
{
"namespace": "com.example.kafka.messageproducer",
"type": "record",
"name": "PersonMessage",
"doc": "Schema for Person validation",
"fields": [
{ "doc": "The firstName of a person.", "name": "firstName","type": "string" },
{ "doc": "The lastName of a person", "name": "lastName","type": "string"},
{ "doc": "The gender", "name": "gender", "type": "string"},
{ "doc": "The age", "name": "age", "type": "int" }
]
}
I have uploaded this avro schema to confluent schema registry and the messagevalidation is working fine.
The kafka documentation is written in AsyncAPI. My goal is to obtain the same avro schema by converting the AsyncApi schema yaml.
Is there any way to convert AsyncAPI yaml file to an Avro schema file?
here is my AsyncAPI file
{
"asyncapi": "2.6.0",
"id": "urn:com:blaise:person",
"info": {
"title": "Person AsynchAPI",
"version": "1.0.0"
},
"servers": {
"non-prod-eu-west-3": {
"url": "pkc-75m1o.europe-west3.gcp.confluent.cloud:9092",
"protocol": "kafka"
}
},
"defaultContentType": "application/json",
"channels": {
"person": {
"publish": {
"message": {
"name": "personupload",
"summary": "Inform the wolrd about the new person created",
"description": "Kafka message related to person\n* publish a message on kafka bus when a new person is created\n",
"contentType": "application/json",
"headers": {
"$ref": "#/components/messageTraits/commonHeaders/headers"
},
"payload": {
"$ref": "#/components/schemas/personPayload"
}
}
}
}
},
"components": {
"messageTraits": {
"commonHeaders": {
"headers": {
"type": "object",
"properties": {
"header1": {
"type": "string",
"description": "the first header"
},
"header2": {
"type": "string",
"description": "the second header"
},
"header3": {
"type": "string",
"description": "the third header"
}
}
}
}
},
"schemas": {
"personPayload": {
"type": "object",
"properties": {
"id": {
"$ref": "#/components/schemas/personId"
},
"firstName": {
"$ref": "#/components/schemas/firstName"
},
"lastName": {
"$ref": "#/components/schemas/lastName"
},
"gender": {
"$ref": "#/components/schemas/gender"
},
"age": {
"$ref": "#/components/schemas/age"
}
},
"required": [
"firstName",
"lastName"
]
},
"personId": {
"type": "string",
"format": "uuid",
"description": "unique uiid to identify a person \nexamples: \n- \"9acabc86-d5e7-41bd-a210-f10bb1984105\n"
},
"firstName": {
"type": "string",
"description": "The firstName of a person \nexamples: \n- Robert\n"
},
"lastName": {
"type": "string",
"description": "The lastName of a person \nexamples: \n- Lafange\n"
},
"gender": {
"type": "string",
"description": "The gender of a persom \n",
"enum": [
"MALE",
"EMALE"
]
},
"age": {
"type": "integer",
"format": "int32",
"description": "The age of a person\n"
}
}
}
}
i was able to convert it to json file using the following tool https://studio.asyncapi.com/. But the result is not a Avro schema at all here is the result:
{
"asyncapi": "2.6.0",
"id": "urn:com:blaise:person",
"info": {
"title": "Person AsynchAPI",
"version": "1.0.0"
},
"servers": {
"non-prod-eu-west-3": {
"url": "pkc-75m1o.europe-west3.gcp.confluent.cloud:9092",
"protocol": "kafka"
}
},
"defaultContentType": "application/json",
"channels": {
"person": {
"publish": {
"message": {
"name": "personupload",
"summary": "Inform the wolrd about the new person
created",
"description": "Kafka message related to person\n*
publish a message on kafka bus when a new person is created\n",
"contentType": "application/json",
"headers": {
"$ref":
"#/components/messageTraits/commonHeaders/headers"
},
"payload": {
"$ref": "#/components/schemas/personPayload"
}
}
}
}
},
"components": {
"messageTraits": {
"commonHeaders": {
"headers": {
"type": "object",
"properties": {
"header1": {
"type": "string",
"description": "the first header"
},
"header2": {
"type": "string",
"description": "the second header"
},
"header3": {
"type": "string",
"description": "the third header"
}
}
}
}
},
"schemas": {
"personPayload": {
"type": "object",
"properties": {
"id": {
"$ref": "#/components/schemas/personId"
},
"firstName": {
"$ref": "#/components/schemas/firstName"
},
"lastName": {
"$ref": "#/components/schemas/lastName"
},
"gender": {
"$ref": "#/components/schemas/gender"
},
"age": {
"$ref": "#/components/schemas/age"
}
},
"required": [
"firstName",
"lastName"
]
},
"personId": {
"type": "string",
"format": "uuid",
"description": "unique uiid to identify a person
\nexamples: \n- \"9acabc86-d5e7-41bd-a210-f10bb1984105\n"
},
"firstName": {
"type": "string",
"description": "The firstName of a person \nexamples: \n- Robert\n"
},
"lastName": {
"type": "string",
"description": "The lastName of a person
\nexamples: \n- Lafange\n"
},
"gender": {
"type": "string",
"description": "The gender of a persom \n",
"enum": [
"MALE",
"EMALE"
]
},
"age": {
"type": "integer",
"format": "int32",
"description": "The age of a person\n"
}
}
}
}
Is there any way to convert AsyncAPI yaml file to an Avro schema file? Thanks a lot.
I don't think you need to convert it. Confluent Schema Registry supports JSON Schema and an AsyncAPI Schema Object is, roughly, a JSON Schema Draft 07 object.
Read more here: https://docs.confluent.io/platform/current/schema-registry/fundamentals/serdes-develop/index.html#serializer-and-formatter.
Let me know if it works :)