Way to represent unknown file size in FTP LIST?

Question

I have an FTP server running on a device which generates a synthetic file. i.e. The file is not actually present on the file system. It is generated on the fly whenever it is requested for download. Its size is not known until it is generated.

Certain FTP clients (e.g. Windows Explorer) are apparently using the info from the FTP LIST command to determine the expected file size after they download it. When I request the download, Windows Explorer is first trying the FTP SIZE extension (RFC 3659), which the server doesn't support. Then it downloads the entire file (I see the whole contents in Wireshark), but it truncates the file to the size that was reported in the LIST.

What is the expected behavior here? Is Windows Explorer acting outside of spec with this optimization / safety measure?

Our current work-around is to show the file size as bigger than it will ever be in the file listing.

Is there a way to represent an unknown file size in an ls -l / FTP LIST? As I read RFC 959, the LIST format is not standardized, but I'm guessing the POSIX ls '-l' format is pretty common, and that Windows Explorer is parsing that.

How should I be handling this situation?

Answer 1

What is the expected behavior here? Is Windows Explorer acting outside of spec with this optimization / safety measure?

The spec does not say anything about checking the file size against the transferred size, but if you strictly follow the spec you don't know if the file was transferred fully. All you can get on the client is that the data connection closed, but this might be also because of problems on the server side. So the client is trying to work around these limits.

Is there a way to represent an unknown file size in an 'ls -l' / FTP LIST? As I read RFC 959, the LIST format is not standardized, but I'm guessing the POSIX format is pretty common, and that Windows Explorer is parsing that.

Yes, there is no standardized format, but commonly the output of ls -l is used. Since this relates to a file system with static files which all have a known file size there is no way to give something like an "unknown" size. But since the format is not standardized you might try to simply give the file list, without any size (like with NLST).

Apart from that your client just implements the typical use case for FTP, i.e. transferring static files. In this case the files sizes are known. But you are using FTP in an atypical way, so you have to deal with these missed expectations and cheat. Since FTP is a nightmare anyway when firewalls or private networks are involved you might better switch to other widely used protocols, like HTTP or WebDAV.

Answer 2

The workaround here for client could be to use wget or 'ftp' command. 'ftp' command example:

1. open console
2. ftp user@yourserver
3. cd to_your_path
4. get your_filename.

Because neither wget nor 'ftp get command' don't need size to download file.

One more solution could be usage of own ftp server. You can find a lot of whose servers with basic features, gui client support and simple code on scripting language, like php. It's simple to change the code for this feature. I have done the same for one of the projects.