I have 5 files of size 200 MB each. I am uploading these files parallel y using Executor service and using TransferManager with multipart threashold = 50 MB and waiting for upload to finish by using blocking method call to upload.waitForCompletion() (which says current thread suspends until upload succeeds or throws error).
Please find below code excerpts:
private static final ExecutorService executor = Executors.newFixedThreadPool(10);
executor.execute(() -> upload("bucketName", new File(fullFilePath)));
public static void upload(final String bucketName, final File filePath) {
AmazonS3 amazonS3 = AmazonS3ClientBuilder.standard().withCredentials(new AWSStaticCredentialsProvider(getCredentals()))
.withRegion(Regions.DEFAULT_REGION).build();
TransferManager tm = TransferManagerBuilder.standard().withS3Client(amazonS3)
.withMultipartUploadThreshold((long) (50 * 1024 * 1025)).build();
final String fileName = filePath.getName();
try {
Upload upload = tm.upload(s3Bucket, fileName, filePath);
upload.waitForCompletion();
log.info("Successfully uploaded file = " + fileName);
} catch (Exception e) {
log.info("Upload failed for file = " + fileName);
log.error(e);
}
}
Main thread does not exits until "Successfully uploaded" is there for all 5 files. Now this program does not throw any error and prints success for all 5 files but when i open the bucket in aws console, nothing is there.
Can anyone suggest what might be happening here or how to further debug it ?
Code is working fine i was looking into the wrong bucket.