Is there a simple method to check if there are changes in a SFTP server?

Question

My objective is to poll the SFTP server for changes. My first thought is to check if the number of files in the dir changed. Then maybe some additional checks for changes in the dir.

Currently I'm using the following:

try {
        FileSystemOptions opts = new FileSystemOptions();
        SftpFileSystemConfigBuilder.getInstance().setStrictHostKeyChecking(opts, "no");
        SftpFileSystemConfigBuilder.getInstance().setUserDirIsRoot(opts, true);
        SftpFileSystemConfigBuilder.getInstance().setTimeout(opts, 60000);

        FileSystemManager manager = VFS.getManager();
        FileObject remoteFile = manager.resolveFile(SFTP_URL, opts);

        FileObject[] fileObjects = remoteFile.getChildren();
        System.out.println(DateTime.now() + " --> total number of files: " + Objects.length);
        for (FileObject fileObject : fileObjects) {
            if (fileObject.getName().getBaseName().startsWith("zzzz")) {
                System.out.println("found one: " + Object.getName().getBaseName());
            }
        }
} catch (Exception e) {
        e.printStackTrace();
}

This is using apache commons vfs2 2.2.0. It works "fine", but when the server has too many files, it takes over minutes just to get the count(currently, it takes over 2 minutes to get the count for a server that has ~10k files). Any way to get the count or other changes on the server faster?

Answer 1

Unfortunately there's no simple way in the SFTP protocol to get the changes. If you can have some daemon running on the server OR if the source of the new files can create/update a helper file, creation of such file with the last modification time in its name or contents can be an option.

Answer 2

I know the SFTP protocol fairly well, having developed commercial SFTP clients and an SFTP server (CompleteFTP), and as far as I know there's no way within the protocol to get a count of files in a directory without listing it. Some servers, such as ours, provide ways of adding custom commands to servers that you can invoke from the client, so it would be possible to add a custom command that returns the number of files in a directory. CompleteFTP also allows you to write custom file-systems so you could potentially write one that only shows files that have changed after a given timestamp when you do a listing, which might be another approach. Our server only runs on Windows though, so that might be show-stopper for you.