I am using using camel-sftp to retrieve some files from a remote SFTP Server and after retrieving them I want to delete them. When I am running my route in Windows files get deleted and everything works as expected. However, when running on CentOS files get retrieved but never deleted and they start piling up in the remote server.
I am using Camel 2.13.1 and Java 7. My consumer URI looks like this:
sftp://remoteUser@remoteHost?privateKeyFile=locationOfMyPrivateKey.key&binary=true&disconnect=true&delay=20s&delete=true&idempotent=false&include=.*.txt&useFixedDelay=false&maxMessagesPerPoll=10&eagerMaxMessagesPerPoll=false&sortBy=reverse:file:name
I have also been looking through the unresolved issues and resolved issues for Camel 2.13.2 and 2.13.3 and just found it this ticket: https://issues.apache.org/jira/browse/CAMEL-7565 which is not exactly what is going on in my scenario. The other possibility is that the issue comes from the underlying library JSCH but camel-ftp is using almost the latest version and the latest version changelog does not mention anything about this.
Last thing, if I use the sftp command in CentOS and I connect remotely to the SFTP Server, I have no issues removing files. This removes the idea that it could be a problem with the key.
Any ideas?
UPDATE
This is the log. It looks like it's not understanding the delete=true option even if it is set. Nothing in the log says 'deleting file', and I guess it is because the thread is not visiting the code on line 382 from SftpOperations.java. Not sure what's going on...
2014-12-15 20:36:28,431 TRACE [Camel (mySFTPRoute) thread #0 - sftp://remoteUser@remoteHost] - GenericFileConsumer.processExchange (GenericFileConsumer.java:356) - Retrieving file: data/file1.txt from: Endpoint[sftp://remoteUser@remoteHost/data?binary=true&delay=20s&delete=true&disconnect=true&eagerMaxMessagesPerPoll=false&idempotent=false&include=.*.txt&maxMessagesPerPoll=10&passiveMode=true&privateKeyFile=locationOfMyPrivateKey.key&reconnectDelay=30000&soTimeout=50000&sortBy=file%3Aname&useFixedDelay=false]
2014-12-15 20:36:28,431 TRACE [Camel (mySFTPRoute) thread #0 - sftp://remoteUser@remoteHost] - SftpOperations.retrieveFile (SftpOperations.java:588) - retrieveFile(data/file1.txt)
2014-12-15 20:36:28,431 TRACE [Camel (mySFTPRoute) thread #0 - sftp://remoteUser@remoteHost] - SftpOperations.getCurrentDirectory (SftpOperations.java:472) - getCurrentDirectory()
2014-12-15 20:36:28,431 TRACE [Camel (mySFTPRoute) thread #0 - sftp://remoteUser@remoteHost] - SftpOperations.getCurrentDirectory (SftpOperations.java:475) - Current dir: /
2014-12-15 20:36:28,431 TRACE [Camel (mySFTPRoute) thread #0 - sftp://remoteUser@remoteHost] - SftpOperations.changeCurrentDirectory (SftpOperations.java:483) - changeCurrentDirectory(data)
2014-12-15 20:36:28,431 TRACE [Camel (mySFTPRoute) thread #0 - sftp://remoteUser@remoteHost] - SftpOperations.changeCurrentDirectory (SftpOperations.java:494) - Compacted path: data -> data using separator: /
2014-12-15 20:36:28,432 TRACE [Camel (mySFTPRoute) thread #0 - sftp://remoteUser@remoteHost] - SftpOperations.getCurrentDirectory (SftpOperations.java:472) - getCurrentDirectory()
2014-12-15 20:36:28,432 TRACE [Camel (mySFTPRoute) thread #0 - sftp://remoteUser@remoteHost] - SftpOperations.getCurrentDirectory (SftpOperations.java:475) - Current dir: /
2014-12-15 20:36:28,432 TRACE [Camel (mySFTPRoute) thread #0 - sftp://remoteUser@remoteHost] - SftpOperations.doChangeDirectory (SftpOperations.java:538) - Changing directory: data
2014-12-15 20:36:28,851 TRACE [Camel (mySFTPRoute) thread #0 - sftp://remoteUser@remoteHost] - IOHelper.copy (IOHelper.java:191) - Copying InputStream: com.jcraft.jsch.ChannelSftp$2@308a9271 -> OutputStream: with buffer: 4096 and flush on each write false
2014-12-15 20:36:29,604 TRACE [Camel (mySFTPRoute) thread #0 - sftp://remoteUser@remoteHost] - SftpOperations.changeCurrentDirectory (SftpOperations.java:483) - changeCurrentDirectory(/)
2014-12-15 20:36:29,604 TRACE [Camel (mySFTPRoute) thread #0 - sftp://remoteUser@remoteHost] - SftpOperations.changeCurrentDirectory (SftpOperations.java:494) - Compacted path: / -> / using separator: /
2014-12-15 20:36:29,605 TRACE [Camel (mySFTPRoute) thread #0 - sftp://remoteUser@remoteHost] - SftpOperations.getCurrentDirectory (SftpOperations.java:472) - getCurrentDirectory()
2014-12-15 20:36:29,605 TRACE [Camel (mySFTPRoute) thread #0 - sftp://remoteUser@remoteHost] - SftpOperations.getCurrentDirectory (SftpOperations.java:475) - Current dir: /data
2014-12-15 20:36:29,605 TRACE [Camel (mySFTPRoute) thread #0 - sftp://remoteUser@remoteHost] - SftpOperations.getCurrentDirectory (SftpOperations.java:472) - getCurrentDirectory()
2014-12-15 20:36:29,605 TRACE [Camel (mySFTPRoute) thread #0 - sftp://remoteUser@remoteHost] - SftpOperations.getCurrentDirectory (SftpOperations.java:475) - Current dir: /data
2014-12-15 20:36:29,605 TRACE [Camel (mySFTPRoute) thread #0 - sftp://remoteUser@remoteHost] - SftpOperations.doChangeDirectory (SftpOperations.java:538) - Changing directory: ..
2014-12-15 20:36:29,816 TRACE [Camel (mySFTPRoute) thread #0 - sftp://remoteUser@remoteHost] - GenericFileConsumer.processExchange (GenericFileConsumer.java:386) - Retrieved file: data/file1.txt from: Endpoint[sftp://remoteUser@remoteHost/data?binary=true&delay=20s&delete=true&disconnect=true&eagerMaxMessagesPerPoll=false&idempotent=false&include=.*.txt&maxMessagesPerPoll=10&passiveMode=true&privateKeyFile=locationOfMyPrivateKey.key&reconnectDelay=30000&soTimeout=50000&sortBy=file%3Aname&useFixedDelay=false]
2014-12-15 20:36:29,817 DEBUG [Camel (mySFTPRoute) thread #0 - sftp://remoteUser@remoteHost] - GenericFileConsumer.processExchange (GenericFileConsumer.java:396) - About to process file: RemoteFile[file1.txt] using exchange: Exchange[file1.txt]
2014-12-15 20:36:29,817 TRACE [Camel (mySFTPRoute) thread #0 - sftp://remoteUser@remoteHost] - DefaultUnitOfWork.<init> (DefaultUnitOfWork.java:77) - UnitOfWork created for ExchangeId: ID-my-machine-name-37091-1418675752051-0-23 with Exchange[file1.txt]
2014-12-15 20:36:29,817 TRACE [Camel (mySFTPRoute) thread #0 - sftp://remoteUser@remoteHost] - EventHelper.doNotifyEvent (EventHelper.java:766) - Notitxtation of event is disabled: ID-my-machine-name-37091-1418675752051-0-23 exchange created: Exchange[file1.txt]
2014-12-15 20:36:29,935 TRACE [Camel (mySFTPRoute) thread #0 - sftp://remoteUser@remoteHost] - GenericFileConsumer$1.done (GenericFileConsumer.java:405) - Done processing file: RemoteFile[file1.txt] synchronously
2014-12-15 20:36:29,935 TRACE [Camel (mySFTPRoute) thread #0 - sftp://remoteUser@remoteHost] - GenericFileConsumer.processExchange (GenericFileConsumer.java:316) - Processing file: RemoteFile[file2.txt]
2014-12-15 20:36:29,937 TRACE [Camel (mySFTPRoute) thread #0 - sftp://remoteUser@remoteHost] - GenericFileConsumer.processExchange (GenericFileConsumer.java:356) - Retrieving file: data/file2.txt from: Endpoint[sftp://remoteUser@remoteHost/data?binary=true&delay=20s&delete=true&disconnect=true&eagerMaxMessagesPerPoll=false&idempotent=false&include=.*.txt&maxMessagesPerPoll=10&passiveMode=true&privateKeyFile=locationOfMyPrivateKey.key&reconnectDelay=30000&soTimeout=50000&sortBy=file%3Aname&useFixedDelay=false]