I'm using AWS SDK for Java v2.0, S3 Async Client with the aws-crt library, which allows to easily put objects with undetermined size as it uses multi-part transfer internally.
Occasionally it fails with this error message, exactly 10 seconds after the putObject call:
The service request was not made within 10 seconds of doBlockingWrite being invoked. Make sure to invoke the service request BEFORE invoking doBlockingWrite if your caller is single-threaded.
I've linked the error to here, but I fail to understand what exactly is wrong.
The S3 Client is initialized as a single instance like so:
S3AsyncClient s3AsyncClient = S3AsyncClient.crtBuilder()
.credentialsProvider(credentialsProvider)
.region(region)
.minimumPartSizeInBytes(8L * 1024 * 1024)
.build();
The application is technically single threaded, but it does process the long task lists via Java streams, although it runs on an AWS Fargate cluster with 0.25 vcpus, so if I read the docs correctly, the fork-join pool will only have a single thread.
Initially I suspected that this is since I have not set up maxConcurrency to be large enough, but the default is 50, which is more than the available CPUs anyway.
I have set maxconcurrency to a high value of 100, but it is hard to verify whether this worked, as the problem occurs only occasionally, and I'd appreciate any insight that is more concrete.