Why is .NET's File.Open with a UNC path making excessive SMB calls?

Question

I have a block of code that needs to open and read a lot of small text files from a NAS server using UNC paths. This code is part of a module that was originally written in C++ but is now being converted to C#. The C# version is significantly slower. I determined that the call to open the file accounts for nearly all of the performance difference. Using WireShark I found that this is because the System.IO.File.Open call makes far more SMB network requests than similar C++ code.

The C++ code makes this call:

FILE *f = _wfsopen(fileName, L"r", _SH_DENYWR);

This results in the following sequence of SMB requests:

NT Create AndX Request, FID: 0x0004, Path: \\a\\i\\a\\q\\~141106162638847.nmd
NT Create AndX Response, FID: 0x0004
Trans2 Request, QUERY_FILE_INFO, FID: 0x0004, Query File Basic Info
Trans2 Response, FID: 0x0004, QUERY_FILE_INFO
Read AndX Request, FID: 0x0004, 1327 bytes at offset 0
Read AndX Response, FID: 0x0004, 1327 bytes
Close Request, FID: 0x0004
Close Response, FID: 0x0004
NT Create AndX Request, FID: 0x0005, Path: \\a\\i\\a\\q\\~141106162638847.nmd
NT Create AndX Response, FID: 0x0005

The C# code makes this call:

FileStream f = File.Open(fileName, FileMode.Open, FileAccess.Read, FileShare.Read);

This results in the following sequence of SMB requests:

Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a\\i\\a\\q\\~141106162638847.nmd
Trans2 Response, QUERY_PATH_INFO
Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path: \\a\\i\\a\\q\\~141106162638847.nmd
Trans2 Response, QUERY_PATH_INFO
Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a\\i\\a\\q\\~141106162638847.nmd
Trans2 Response, QUERY_PATH_INFO
Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: 
Trans2 Response, QUERY_PATH_INFO
Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path: 
Trans2 Response, QUERY_PATH_INFO
Trans2 Request, FIND_FIRST2, Pattern: \\a
Trans2 Response, FIND_FIRST2, Files: a
Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a
Trans2 Response, QUERY_PATH_INFO
Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path: \\a
Trans2 Response, QUERY_PATH_INFO
Trans2 Request, FIND_FIRST2, Pattern: \\a\\i
Trans2 Response, FIND_FIRST2, Files: i
Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a\\i
Trans2 Response, QUERY_PATH_INFO
Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path: \\a\\i
Trans2 Response, QUERY_PATH_INFO
Trans2 Request, FIND_FIRST2, Pattern: \\a\\i\\a
Trans2 Response, FIND_FIRST2, Files: a
Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a\\i\\a
Trans2 Response, QUERY_PATH_INFO
Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path: \\a\\i\\a
Trans2 Response, QUERY_PATH_INFO
Trans2 Request, FIND_FIRST2, Pattern: \\a\\i\\a\\q
Trans2 Response, FIND_FIRST2, Files: q
Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a\\i\\a\\q\\~141106162638847.nmd
Trans2 Response, QUERY_PATH_INFO
Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path: \\a\\i\\a\\q\\~141106162638847.nmd
Trans2 Response, QUERY_PATH_INFO
Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a\\i\\a\\q\\~141106162638847.nmd
Trans2 Response, QUERY_PATH_INFO
Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: 
Trans2 Response, QUERY_PATH_INFO
Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path: 
Trans2 Response, QUERY_PATH_INFO
Trans2 Request, FIND_FIRST2, Pattern: \\a
Trans2 Response, FIND_FIRST2, Files: a
Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a
Trans2 Response, QUERY_PATH_INFO
Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path: \\a
Trans2 Response, QUERY_PATH_INFO
Trans2 Request, FIND_FIRST2, Pattern: \\a\\i
Trans2 Response, FIND_FIRST2, Files: i
Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a\\i
Trans2 Response, QUERY_PATH_INFO
Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path: \\a\\i
Trans2 Response, QUERY_PATH_INFO
Trans2 Request, FIND_FIRST2, Pattern: \\a\\i\\a
Trans2 Response, FIND_FIRST2, Files: a
Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a\\i\\a
Trans2 Response, QUERY_PATH_INFO
Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path: \\a\\i\\a
Trans2 Response, QUERY_PATH_INFO
Trans2 Request, FIND_FIRST2, Pattern: \\a\\i\\a\\q
Trans2 Response, FIND_FIRST2, Files: q
Close Request, FID: 0x000f
Close Response
NT Create AndX Request, FID: 0x0018, Path: \\a\\i\\a\\q\\~141106162638847.nmd
NT Create AndX Response, FID: 0x0018
Trans2 Request, QUERY_FILE_INFO, FID: 0x0018, Query File Basic Info
Trans2 Response, FID: 0x0018, QUERY_FILE_INFO
Read AndX Request, FID: 0x0018, 1327 bytes at offset 0
Read AndX Response, FID: 0x0018, 1327 bytes
Close Request, FID: 0x0018
Close Response, FID: 0x0018
NT Create AndX Request, FID: 0x0019, Path: \\a\\i\\a\\q\\~141106162638847.nmd
NT Create AndX Response, FID: 0x0019

Why does System.IO.File.Open make all these extra SMB requests? Is there any way to change this code to avoid all these extra requests?

Answer 1

In short, File.Open calls new FileStream() and new FileStream() does a lot of calls:

1. NormalisePath.

   String filePath = Path.NormalizePath(path, true, maxPath); // fullCheck: true
   

leads to this code:

1.a: Get full path:

    if (fullCheck) { ... 
        result = newBuffer.GetFullPathName();

GetFullPathName() calls Win32Native.GetFullPathName one or two times (depending on the lentgh of resulting path).

1.b. Trying to expand short path. Your path contains ~ char, so it looks like a candidate for a path expanding:

    if (mightBeShortFileName) {
        bool r = newBuffer.TryExpandShortFileName();

as a result, Win32Native.GetLongPathName() is called.

2. FileIoPermission.Demand() (for non-trusted only):

   // All demands in full trust domains are no-ops, so skip 
   if (!CodeAccessSecurityEngine.QuickCheckForAllDemands()) {
       ...
       new FileIOPermission(secAccess, control, new String[] { filePath }, false, false).Demand();
   

3. Open fileStream (floppy strikes back;)):

    // Don't pop up a dialog for reading from an emtpy floppy drive
   int oldMode = Win32Native.SetErrorMode(Win32Native.SEM_FAILCRITICALERRORS);
   try {
       ...
       _handle = Win32Native.SafeCreateFile(tempPath, fAccess, share, secAttrs, mode, flagsAndAttributes, IntPtr.Zero);
   

4. Win32Native.GetFileType()

Not all of them would lead to smb request, but some will do. I've tried to reproduce chatty requests by debugging with source step-by-step (here's manual for enabling the .net source debugging) and checking the log after each step. Resuts are more similar to your's first listing. If you're really interested in finding the real issue, you'll have to do it yourself.

UPD Note that I've checked current (.net 4.5.2) behavior. It was changed multiple times since 2.0 (e.g. FileIOPermission.Demand() originally was called for full-trusted code too), so it depends:)

Answer 2

I don't really have a specific answer to why the .NET implementation is so chatty, but this behaviour would be due to the implementation of System.IO.FileStream as all that File.Open(fileName, FileMode.Open, FileAccess.Read, FileShare.Read); is doing is passing the parameters to the FileStream constructor.

public static FileStream Open(string path, FileMode mode, FileAccess access, FileShare share)
{
    return new FileStream(path, mode, access, share);
}

Changing the behaviour of FileStream would mean that you would basically have to re-implement the FileStream class which will require a lot of effort.

Your other more simpler alternative would be to create a native wrapper that calls the C++ code you gave. Then call the native wrapper from your C# code.