AWS SDK for Java v2.0 - using S3 CRT Async Client, s3Client.putObject fails with InvalidStateException after 10 seconds of hanging

1k views Asked by At

I'm using AWS SDK for Java v2.0, S3 Async Client with the aws-crt library, which allows to easily put objects with undetermined size as it uses multi-part transfer internally.

Occasionally it fails with this error message, exactly 10 seconds after the putObject call:

The service request was not made within 10 seconds of doBlockingWrite being invoked. Make sure to invoke the service request BEFORE invoking doBlockingWrite if your caller is single-threaded.

I've linked the error to here, but I fail to understand what exactly is wrong.

The S3 Client is initialized as a single instance like so:

S3AsyncClient s3AsyncClient = S3AsyncClient.crtBuilder()
                    .credentialsProvider(credentialsProvider)
                    .region(region)
                    .minimumPartSizeInBytes(8L * 1024 * 1024)
                    .build();

The application is technically single threaded, but it does process the long task lists via Java streams, although it runs on an AWS Fargate cluster with 0.25 vcpus, so if I read the docs correctly, the fork-join pool will only have a single thread.

Initially I suspected that this is since I have not set up maxConcurrency to be large enough, but the default is 50, which is more than the available CPUs anyway.

I have set maxconcurrency to a high value of 100, but it is hard to verify whether this worked, as the problem occurs only occasionally, and I'd appreciate any insight that is more concrete.

0

There are 0 answers