I have tried to use Apache Kafka Connect with MongoDB sink connector (connector)
When I used Avro format it worked except of one issue, I had to create the topic with one partition because the connector uses record.kafkaOffset() for the _id of the new Mongo record (so with multi partitions I get the same id for different records).
How can I fix it?
I would like to test it with Json so I created a new topic for that. And changed the converter configurations to JsonConverter. When I run it I get the following error:
java.lang.ClassCastException: java.util.HashMap cannot be cast to org.apache.kafka.connect.data.Struct at org.apache.kafka.connect.mongodb.MongodbSinkTask.put(MongodbSinkTask.java:106) at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:280) at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:176) at org.apache.kafka.connect.runtime.WorkerSinkTaskThread.iteration(WorkerSinkTaskThread.java:90) at org.apache.kafka.connect.runtime.WorkerSinkTaskThread.execute(WorkerSinkTaskThread.java:58) at org.apache.kafka.connect.util.ShutdownableThread.run(ShutdownableThread.java:82)
Does this connector work with Json?