Delphi Bug in Indy FTP List method?

Question

I'm trying to generate a list of files matching a certain file mask and Indy falls over with this error

EidReplyRFCError with message '.': No such file or directory.

I've tried several variations and this is the result:

FTP.List( aFiles, '', true ); => this works

FTP.List( aFiles, '*.*', false ); => this works too

FTP.List( aFiles, '*.*', true ); => this fails

FTP.List( aFiles, '*.zip', true ); => this fails too (despite it being the example in the latest documentation)

FTP.List( '*.*', false ); => this works

FTP.List( '*.*', true ); => this fails

I'm using Delphi XE5 & Indy version 10.6. The same issue exists in XE8 if relevant.

Maybe the functionality has changed and the documentation is now wrong or it's a bug in Indy?

I need the "details" so I can compare timestamps & sizes too.

Answer 1

This is not a bug in TIdFTP. It is more an omission in the Indy documentation.

EIdReplyRFCError means the FTP server itself is reporting an error in response to the command that TIdFTP.List() is sending. Depending on the values of the ADetails parameter and TIdFTP's UseMLIS+CanUseMLS properties, List() can send one of three different commands:

ADetails=False:

    NLST [ASpecifier]

ADetails=True:

  TIdFTP.UseMLIS=True and TIdFTP.CanUseMLS=True:

    MLSD [ASpecifier]

  TIdFTP.UseMLIS=False or TIdFTP.CanUseMLS=False:

    LIST [ASpecifier]

Thus:

FTP.List( aFiles, '', true ); // this works
// sends either 'LIST' or 'MLSD'

FTP.List( aFiles, '*.*', false ); // this works too
// sends 'NLST *.*'

FTP.List( aFiles, '*.*', true ); // this fails
// sends either 'LIST *.*' or 'MLSD *.*'

FTP.List( aFiles, '*.zip', true ); // this fails too
// sends either 'LIST *.zip' or 'MLSD *.zip'

FTP.List( '*.*', false ); // this works
// sends 'NLST *.*'

FTP.List( '*.*', true ); // this fails
// sends either 'LIST *.*' or 'MLSD *.*'

Note that all of the commands that "fail" have something in common - they might be sending an MLSD ASpecifier command.

Per RFC 959, which defines the LIST and NLST commands:

LIST (LIST)

This command causes a list to be sent from the server to the passive DTP. If the pathname specifies a directory or other group of files, the server should transfer a list of files in the specified directory. If the pathname specifies a file then the server should send current information on the file. A null argument implies the user's current working or default directory. ...

NAME LIST (NLST)

This command causes a directory listing to be sent from server to user site. The pathname should specify a directory or other system-specific file group descriptor; a null argument implies the current directory. ...

Per RFC 3659, which defines the MLSD command:

The MLST and MLSD commands each allow a single optional argument. This argument may be either a directory name or, for MLST only, a file name. For these purposes, a "file name" is the name of any entity in the server NVFS which is not a directory. Where TVFS is supported, any TVFS relative pathname valid in the current working directory, or any TVFS fully qualified pathname, may be given. If a directory name is given then MLSD must return a listing of the contents of the named directory, otherwise it issues a 501 reply, and does not open a data connection. ...

If no argument is given then MLSD must return a listing of the contents of the current working directory, and MLST must return a listing giving information about the current working directory itself. ...

...

If the Client-FTP sends an invalid argument, the server-FTP MUST reply with an error code of 501.

*.* and *.zip are not directory names, thus the server will fail if TIdFTP.List() sends an MLSD *.* or MLSD *.zip command. So it stands to reason that TIdFTP.UseMLIS and TIdFTP.CanUseMLS are likely both True in your case (UseMLIS is True by default, and CanUseMLS is commonly True on modern FTP servers).

The MLSD command does not support server-side filtering like the LIST/NLST commands do. So you cannot use things like *.* and *.zip with MLSD. You would have to retrieve the full directory listing and then ignore any entries that you are not interested in. Otherwise, set TIdFTP.UseMLIS to False before calling TIdFTP.List(), but then you run the risk of TIdFTP.DirectoryListing incorrectly parsing the directory listing of some servers, as the format used by the LIST command was never standardized, and there are hundreds of custom formats being used all over the Internet (and why TIdFTP in Indy 10 includes dozens of listing parsers when LIST is used). Unlike MLSx, which has a standardized format (which is why it was introduced in the first place, to replace the shortcomings of LIST).

Thus, what this all comes down to is - when TIdFTP.UseMLIS and TIdFTP.CanUseMLS are both True, ASpecifier MUST be blank or a directory, NOT a file mask.

The TIdFTP.List() documentation does state that List() may internally call TIdFTP.ExtListDir() to send an MLSD command, but it does not specifically mention this particular restriction on the ASpecifier parameter in that case:

If CanUseMLS contains True, the ExtListDir is called to capture and store the results of the FTP MLSD command in the ADest parameter variable instead of the LIST or NLST commands. No additional processing is performed in the List method under this circumstance, and the method is exited.

When ADetails is False, only the file or directory name is returned in the ADest string list using the FTP NLST command. When ADetails is True, List can return FTP server-dependent details including the file size, date modified, and file permissions for the Owner, Group, and User using the FTP LIST command.

The TIdFTP.ExtListDir() documentation does state that its input parameter must be a directory name, though:

The MLSD command, supported in ExtListDir, accepts an optional directory name or relative path in Adirectory for the directory listing. If am empty string is passed in ADirectory, the current directory is used for the directory listing operation.

On a side note: the TIdFTP.DirFormat property will tell you which listing format was detected after TIdFTP.DirectoryListing has parsed the results. Or you can look at the Details and UsedMLS properties of TListFTP.ListResult (type-cast it to TIdFTPListResult to access the properties) to deduce which command was sent by TIdFTP.List() (if successful).

Answer 2

An alternative solution is to include

IdAllFTPListParsers

In your uses clause and to disable UseMLIS.

Like so:

uses
  ....
  IdAllFTPListParsers;
   .....
procedure TForm1.DoThis;
var
  i: integer;
begin
  if not IDFTP1.Connected then IDFTP1.Connect;
  IDFTP1.UseMLIS:= false;
  IDFTP1.List;
  for i:= 0 to IDFTP1.DirectoryListing.Count -1 do begin
    .. process directory items.
    IdFTP1.TransferType:= ftBinary;
    ..Get your files