Avro SchemaStore & versioning without a central registry

676 views Asked by At

I'm relatively new to Avro but I'm fairly certain that I've read and experimented enough to understand how it generally works.

I'm working on a Java distributed system where the Avro schemas are defined by a single consumer. The schemas are used to generate the POJOs which are then imported by the producer applications. Due to variable release cadence of our system, different producer applications can be generating messages using old versions of the schema - this is why we chose Avro.

Due to the way we define and generate the schemas in a single Maven module, there's no real need for us to deploy a full-scale registry. We have no use-case where we need to dynamically look up a schema for either serialisation or deserialisation.

All we actually need is a strategy (and tooling) which will allow us to write an implementation of SchemaStore which is backed by schema files on the classpath. The module can back bundled into a Maven package and imported into the producer apps.

Is anyone aware of any methods or tooling which allow us to manage multiple versions of schema files, track version numbers, and automatically generate the POJO classes?

I'm imagining something that works a bit like the old Maven version plugin. We have the "current" version of the schema committed in code. When we want to make a change, we change this file and then run the plugin which automatically commits it as a new version and generates the new POJO. The old version is maintained and also committed in code so that it remains available to the SchemaStore.

It's worth noting that we're not using Kafka or any other kind of streaming system. All messages are sent using plain-old REST endpoints and are serialised using single-object-encoding (so they contain the schema fingerprint)

I hope this all makes sense. Any help would be much appreciated!

0

There are 0 answers