S3TransferManager memory leak?

438 views Asked by At

I'm attempting to use the AWS S3TransferManager in bom:2.20.26, but there appears to be a memory leak. Perhaps I am using the API incorrectly? The previous S3TransferManager had a shutdownNow( ) method, but this version only has close( ) that I can see. The test code below is a loop downloading a 30M file with a 5 sec sleep. If I run this, over time the memory usage shown by top keeps growing. Thank you if someone can confirm this or tell me what I am doing incorrectly.

My java file:

package com.s3tmtest;
import software.amazon.awssdk.transfer.s3.S3TransferManager;
import software.amazon.awssdk.transfer.s3.model.DownloadFileRequest;
import software.amazon.awssdk.transfer.s3.model.FileDownload;

import java.nio.file.Paths;

public class Main {
    public static void log(String s){
        System.out.println(s);
    }
    public static void main(String args[]){
        String bucket="your_bucket_name";
        String key="your_large_file";
        String outFile="your_output_filename";
        int counter=0;
        while(true) {
            S3TransferManager transferManager = S3TransferManager.create();

            DownloadFileRequest downloadFileRequest =
                    DownloadFileRequest.builder()
                            .getObjectRequest(req -> req.bucket(bucket).key(key))
                            .destination(Paths.get(outFile))
                            .build();

            FileDownload download = transferManager.downloadFile(downloadFileRequest);

            // Wait for the transfer to complete
            download.completionFuture().join();
            log("count="+(++counter));
            try {
                Thread.sleep(5000);
            }catch(Exception e){
                log(e.toString());
            }finally{
                transferManager.close();
                transferManager=null;
            }
        }
    }
}

My Gradle file:

plugins {
    id 'java'
    id 'application'
    id 'com.github.johnrengelman.shadow' version '7.1.2'
}

compileJava {
    options.release = 17
}
mainClassName='com.s3tmtest.Main'
group 'com.s3tmtest'
version '1.0'
archivesBaseName='s3tmtest'
repositories {
    google()
    mavenCentral()
}

dependencies {
    implementation platform('software.amazon.awssdk:bom:2.20.26')
    implementation 'software.amazon.awssdk:s3'
    implementation 'software.amazon.awssdk:aws-crt-client'
    implementation 'software.amazon.awssdk:s3-transfer-manager'
}
0

There are 0 answers