We are receiving ProvisionedThroughputExceededExceptions on the getShardIterator request while consuming from a Kinesis shard. We noticed in the logs that the error occurs on a getShardIterator request being sent before a checkpoint occurs. AWS support believes the getShardIterator in KCL is being called explicitly by the Spring code. Can you tell us if and why the getShardIterator would be called multiple times per second per shard while processing messages? Note that we are using Spring Cloud Stream Binder for Kinesis version 2.2.0.
Here's the reply we received from the AWS Kinesis team providing support to us:
From the logs provided, I see that the ProvisonedThroughputExceededException errors are related to the GetShardIterator calls. When you read repeatedly from a stream, you use a GetShardIterator request to get the first shard iterator for use in your first GetRecords request and for subsequent reads use the shard iterator returned by the GetRecords request in NextShardIterator. A new shard iterator is returned by every GetRecords request in NextShardIterator, which you use in the ShardIterator parameter of the next GetRecords request. If a GetShardIterator request is made too often, you receive a ProvisionedThroughputExceededException, which we see happening here. GetShardIterator has a limit of five transactions per second per account per open shard.
Every GetRecords call should not use the GetShardIterator API. Instead, it should use the previous GetRecords requests NextShardIterator response. Please note we are in discussion with the internal team on further guidance regarding these ProvisonedThroughputExceededException related to the GetShardIterator API.