KCL application does not take all shards while consuming multiple streams

342 views Asked by At

I have a consumer application developed using KCL. application creates a Worker for each stream and each stream has more than one shard. In different executions, application takes different number of shards and it is not taking all shards. I am running only one instance of the application and I expect to consume all shards. Where could be a problem so I can look to fix it?

Thank you

1

There are 1 answers

0
sansari On

As of now with V.1 and V.2 KCL is working per stream and it is not possible to consume multiple streams with one KCL application. There is an open ticket to be able running multiple instance on same JVM. Please check out KCL Github conversation for more information on