File in use by another process error when uploading a file using server but not localhost

Question

I am fairly new to MVC and .NET, and I've hit my first problem that I have been stuck on for 3 days now. I apologize for the lengthy post.

Part of the project I am working on requires that the user be able to select an XLS or XLSX file and upload it so the data can be imported by the application. It is likely the files being uploaded will have upwards of 20,000 rows of data.

I have test files with 5000, 10000, and 20000 rows in them. When I run my app on my local machine (using Visual Studio 2010), all of these files get saved to the network share and processed just fine. No errors. However, when I deploy the app to our development server, I get an error after 5 minutes that the 10k and 20k file cannot be accessed because it is in use by another process.

Error message: The process cannot access the file 'removed this part of path\TestBook-10k-rows.xlsx' because it is being used by another process.

Stack trace:

at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath) 
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost) 
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, String msgPath, Boolean bFromProxy) 
at System.IO.FileStream..ctor(String path, FileMode mode) 
at System.Web.HttpPostedFile.SaveAs(String filename) 
at System.Web.HttpPostedFileWrapper.SaveAs(String filename) 
at edtStar.Controllers.TableController.Import(HttpPostedFileBase UploadFile, FormCollection collection) in <removed this part of the path>\TableController.cs:line 392

The 10000 row file is 304KB in size, and I have the length limits set as follows in my web.config:

<httpRuntime 
  maxRequestLength="4096"
  requestLengthDiskThreshold="1024"/>

I can't find anything special about 5 minutes, but it is always returning the error 5 minutes after I start the upload.

I have tried this with Chrome and IE. Both work via localhost, neither work via our dotnet app server.

The stack trace says it is blowing up on the SaveAs() method, but I can see the file on the network share location and the size matches.

After saving the file, I have to read it and return the data to a new view. There is a fair amount of processing being done on the data before returning it to the view, which is where I would expect to wait 5 minutes or more. After I'm done reading the file, I close the connection and delete the file from the network share. The file gets deleted shortly after I get the exception, too. The file does not exist when I start each upload.

I am the only person working with this application right now as it is still in development. I do not believe anyone else is accessing the saved copy on the network share while I am doing my testing. I can consistently reproduce this issue but I have no idea why it is happening.

Has anyone seen anything like this or have any suggestions for me? My guess is there's a setting somewhere on our app server but I haven't been able to pinpoint it.

Thanks!

Edit:

Here is the code that is handling the file upload.

// Validate, save, and read the selected file.
[AcceptVerbs(HttpVerbs.Post)]
public ActionResult Import(HttpPostedFileBase UploadFile, FormCollection collection)
{
  // Do some initial file validation - removed

  string filePath = Path.Combine(EDTConstants.NETWORK_SHARE_PATH, Path.GetFileName(UploadFile.FileName));

  try
  {
    // Do some validation on the file that was uploaded.
    if (UploadFile == null)
    {
      // return an error here - there was no file selected.
    }

    if (UploadFile.ContentLength == 0)
    {
      // return an error here - the file is empty.
    }

    if (!filePath.ToUpper().EndsWith("XLS") && !filePath.ToUpper().EndsWith("XLSX"))
    {
      // return an error here - the file extension is not supported.
    }

    // Things are good so far.  Save the file so we can read from it.
    UploadFile.SaveAs(filePath);
    DataSet fileDS = new DataSet();
    string connString = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + filePath + ";Extended Properties=\"Excel 12.0;HDR=YES;\"";
    using (OleDbConnection conn = new OleDbConnection(connString))
    {
      conn.Open();
      using (DataTable dtExcel = conn.GetSchema("Tables"))
      {
        // ... a bunch of code removed ...
      }
      conn.Close();
    }

    CloseFile(filePath);
  }
  catch (Exception e)
  {
    // Unexpected error
    EDTUtils.LogErrorMessage(EDTConstants.TABLE_IMPORT, User.Identity.Name, e);
    ViewData[EDTConstants.SSN_VD_ERROR_MSG] = EDTConstants.EM_UNEXPECTED + System.Environment.NewLine + e.Message;
    ViewData[EDTConstants.VIEWDATA_FILE] = UploadFile.FileName;
    CloseFile(filePath);
    return View(tab);
  }

// Closes a file - logs errors but does not throw any exceptions.
private void CloseFile(string filePath)
{
  try
  {
    System.IO.File.Delete(filePath);
  }
  catch (IOException ioe)
  {
    EDTUtils.LogErrorMessage(EDTConstants.TABLE_IMPORT, User.Identity.Name, ioe);
  }
}

I do see the same error logged, once when doing the SaveAs() and then again when doing the Delete. It's weird, though, the file does go away. It's almost like the app server has another thread trying to do something during this process, and that doesn't happen on localhost.

Edit

After I changed the executionTimeout value, I noticed that Chrome said "Uploading 79%..." at the 5 minute mark, then the file disappeared on the network share, then reappeared, and then after a minute or two, Chrome said the page was not available, with "Error code: ERR_INVALID_RESPONSE". The file has not been deleted from the network share yet. In IE, I am asked to log in to the app again at the 5 minute mark.

Answer 1

maxRequestLength is specified in KB but requestLengthDiskThreshold specifies the limit for the input stream buffering threshold, in bytes.

Therefore you need to change requestLengthDiskThreshold to 543744 (531KB x 1024)

Also check any of the following settings that apply to your scenario

<httpRuntime
   executionTimeout = "HH:MM:SS" 
   maxRequestLength = "number" 
   requestLengthDiskThreshold = "number" 
   useFullyQualifiedRedirectUrl = "[True|False]" 
   minFreeThreads = "number" 
   minLocalRequestFreeThreads = "number" 
   appRequestQueueLimit = "number"
   enableKernelOutputCache = "[True|False]" 
   enableVersionHeader = "[True|False]" 
   apartmentThreading = "[True|False]"
   requireRootedSaveAsPath = "[True|False]"
   enable = "[True|False]" 
   sendCacheControlHeader = "[True|False]" 
   shutdownTimeout = "HH:MM:SS"
   delayNotificationTimeout = "HH:MM:SS"
   waitChangeNotification = "number" 
   maxWaitChangeNotification = "number" 
   enableHeaderChecking = "[True|False]" 
/>

Maybe a timeout is happening for the request, and the file is kept by the upload process and in the next attempt is still busy.

I suggest you delete any of the uploaded files, recycle you application pool, set a executionTimeout in the httpRuntime settings and try again.

The details of the httpRuntime settings are here http://msdn.microsoft.com/en-us/library/e1f13641%28v=vs.85%29.aspx

[AcceptVerbs(HttpVerbs.Post)]
public ActionResult Import(HttpPostedFileBase UploadFile, FormCollection collection)
{
  // Do some initial file validation - removed

  string filePath = Path.Combine(EDTConstants.NETWORK_SHARE_PATH, Path.GetFileName(UploadFile.FileName));

  try
  {
    // Do some validation on the file that was uploaded.
    if (UploadFile == null)
    {
      // return an error here - there was no file selected.
    }

    if (UploadFile.ContentLength == 0)
    {
      // return an error here - the file is empty.
    }

    if (!filePath.ToUpper().EndsWith("XLS") && !filePath.ToUpper().EndsWith("XLSX"))
    {
      // return an error here - the file extension is not supported.
    }

    // Things are good so far.  Save the file so we can read from it.
    UploadFile.SaveAs(filePath);
    DataSet fileDS = new DataSet();
    string connString = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + filePath + ";Extended Properties=\"Excel 12.0;HDR=YES;\"";
    using (OleDbConnection conn = new OleDbConnection(connString))
    {
      conn.Open();
      using (DataTable dtExcel = conn.GetSchema("Tables"))
      {
        // ... a bunch of code removed ...
      }
      conn.Close();
    }

  }
  catch (Exception e)
  {
    // Unexpected error
    EDTUtils.LogErrorMessage(EDTConstants.TABLE_IMPORT, User.Identity.Name, e);
    ViewData[EDTConstants.SSN_VD_ERROR_MSG] = EDTConstants.EM_UNEXPECTED + System.Environment.NewLine + e.Message;
    ViewData[EDTConstants.VIEWDATA_FILE] = UploadFile.FileName;


  }
  finally
  {
    CloseFile(filePath);
  }
  //you were only returning in the CATCH ? 
  return View(tab);

}

Also check that there is no ANTIVIRUS trying to scan the files that you just uploaded