Download entire FTP directory in Java (Apache Net Commons)

Question

I am trying to recursively iterate through the entire root directory that I arrive at after login to the FTP server.

I am able to connect, all I really want to do from there is recurse through the entire structure and and download each file and folder and have it in the same structure as it is on the FTP. What I have so far is a working download method, it goes to the server and gets my entire structure of files, which is brilliant, except it fails on the first attempt, then works the second time around. The error I get is as follows:

java.io.FileNotFoundException: output-directory\test\testFile.png (The system cannot find the path specified)

I managed to do upload functionality of a directory that I have locally, but can't quite get downloading to work, after numerous attempts I really need some help.

public static void download(String filename, String base)
{
    File basedir = new File(base);
    basedir.mkdirs();

    try
    {
        FTPFile[] ftpFiles = ftpClient.listFiles();
        for (FTPFile file : ftpFiles)
        {
            if (!file.getName().equals(".") && !file.getName().equals("..")) {
                // If Dealing with a directory, change to it and call the function again
                if (file.isDirectory())
                {
                    // Change working Directory to this directory.
                    ftpClient.changeWorkingDirectory(file.getName());
                    // Recursive call to this method.
                    download(ftpClient.printWorkingDirectory(), base);

                    // Create the directory locally - in the right place
                    File newDir = new File (base + "/" + ftpClient.printWorkingDirectory());
                    newDir.mkdirs();

                    // Come back out to the parent level.
                    ftpClient.changeToParentDirectory();
                }
                else
                {
                    ftpClient.setFileType(FTPClient.BINARY_FILE_TYPE);
                    String remoteFile1 = ftpClient.printWorkingDirectory() + "/" + file.getName();
                    File downloadFile1 = new File(base + "/" + ftpClient.printWorkingDirectory() + "/" + file.getName());
                    OutputStream outputStream1 = new BufferedOutputStream(new FileOutputStream(downloadFile1));
                    boolean success = ftpClient.retrieveFile(remoteFile1, outputStream1);
                    outputStream1.close();
                }
            }
        }
    }
    catch(IOException ex)
    {
        System.out.println(ex);
    }
}

Answer 1

Your problem (well, your current problem after we got rid of the . and .. and you got past the binary issue) is that you are doing the recursion step before calling newDir.mkdirs().

So suppose you have a tree like

.
..
someDir
   .
   ..
   someFile.txt
someOtherDir
   .
   ..
someOtherFile.png

What you do is skip the dot files, see that someDir is a directory, then immediately go inside it, skip its dot files, and see someFile.txt, and process it. You have not created someDir locally as yet, so you get an exception.

Your exception handler does not stop execution, so control goes back to the upper level of the recursion. At this point it creates the directory.

So next time you run your program, the local someDir directory is already created from the previous run, and you see no problem.

Basically, you should change your code to:

            if (file.isDirectory())
            {
                // Change working Directory to this directory.
                ftpClient.changeWorkingDirectory(file.getName());

                // Create the directory locally - in the right place
                File newDir = new File (base + "/" + ftpClient.printWorkingDirectory());
                newDir.mkdirs();

                // Recursive call to this method.
                download(ftpClient.printWorkingDirectory(), base);

                // Come back out to the parent level.
                ftpClient.changeToParentDirectory();
            }

Answer 2

A complete standalone code to download all files recursively from an FTP folder:

private static void downloadFolder(
    FTPClient ftpClient, String remotePath, String localPath) throws IOException
{
    System.out.println("Downloading folder " + remotePath + " to " + localPath);

    FTPFile[] remoteFiles = ftpClient.listFiles(remotePath);

    for (FTPFile remoteFile : remoteFiles)
    {
        if (!remoteFile.getName().equals(".") && !remoteFile.getName().equals(".."))
        {
            String remoteFilePath = remotePath + "/" + remoteFile.getName();
            String localFilePath = localPath + "/" + remoteFile.getName();

            if (remoteFile.isDirectory())
            {
                new File(localFilePath).mkdirs();

                downloadFolder(ftpClient, remoteFilePath, localFilePath);
            }
            else
            {
                System.out.println("Downloading file " + remoteFilePath + " to " +
                    localFilePath);

                OutputStream outputStream =
                    new BufferedOutputStream(new FileOutputStream(localFilePath));
                if (!ftpClient.retrieveFile(remoteFilePath, outputStream))
                {
                    System.out.println("Failed to download file " + remoteFilePath);
                }
                outputStream.close();
            }
        }
    }
}