ftp download for small datasets works but for more than 300KB not

35 views Asked by At

I am trying to download a file from my ftp server. As soon as the file is larger than 300KB, the download fails. Below that limit, it works. The ftp server is running in a Docker container (image: delfer/alpine-ftp-server). Here is my method:

public InputStream getInputStreamFromURI(String uri) throws IOException {
        try {
            ftp.addProtocolCommandListener(new PrintCommandListener(
                    new PrintWriter(new 
OutputStreamWriter(System.out, "UTF-8")), true));
            assureConnection(false);
            ftp.setBufferSize(2048000); 
            ftp.setAutodetectUTF8(true);
            ftp.setDataTimeout(Duration.ofSeconds(6000));
            ftp.enterLocalPassiveMode();
            InputStream is;
            logger.trace("Retrieve file " + uri + " from FTP server...");
            is = ftp.retrieveFileStream(uri);
            ftp.completePendingCommand();
            logger.trace("FTP response was: " + ftp.getReplyString());          
            return is;
        
        } catch (FTPConnectionClosedException e1) {
            logger.error("Server has closed connection, we will try again...");
            return getInputStreamFromURI(uri);
        } catch (ConnectException e2) {
            logger.error("The server could not be connected, let's try again...");
            return getInputStreamFromURI(uri);
        } catch (NoRouteToHostException e3) {
            logger.error("Server was not routable, let's try again...");
            return getInputStreamFromURI(uri);
        }   
    }

Here are the logs from the ftp container:

[pid 1336] OK LOGIN: Client "172.12.123.12"
[pid 1338] [ppdw] FAIL DOWNLOAD: Client "172.12.123.12", "file.csv", 366760 byte

What could be the problem?

So far I have tried to save the file temporarily. However, this temp file is not created. Another idea would be to split the file, but that didn't work either.

Update:

Here is the log for the small file:

OK DOWNLOAD: Client "172.12.123.12", "file.csv", 228893 bytes, 0.74Kbyte/sec

Update#2: Adding the "ftp.addProtocolCommandListener" did not change anything in the logs of ftp container. Only the application logs show now:

Log in configured FTP server...
NOOP
200 NOOP ok.
PASV
227 Entering Passive Mode (172,12,123,98,82,17).
RETR files/config/tpch1.properties
150 Opening BINARY mode data connection for file.properties (842 bytes).
226 Transfer complete.
0

There are 0 answers