whats the best way to consume and commit by batch with NATS Jetstream

Question

I need to consume a stream from Jetstream by batch and store bulk in database with acknowledgment after database bulk insert. At this time I use pull consumer, store all messages and commit 1 by 1 on successful bulk insert. Do I need to ack all messages? I saw that I can commit the latest message but for this I need to be in consumer push mode.

Answer 1

ConsumerConfiguration.AckWait is probably what you need.

For pull mode, it is required that you ack every message. You could set the AckWait to a longer duration, which should give you enough time to make sure your bulk store works and then go back and individually ack the messages.

If you use push mode, you could use AckPolicy.All and set the AckWait longer also. You can then read messages in push mode, (which looks like one message at a time, but the server has actually pushed many messages to the client and the client is buffering them.) Once you've collected the number of messages you want to write to the database, you can write them in bulk, then you can ack the last message when you know the database write has succeeded.