How to handle 'missing' or 'wrong file names' urls when using OWIN Static Files?

Question

I've successfully used Static Files in my asp.net web api project. It is serving files that exist perfectly. The problem is when the file no longer exists (got deleted) or it is renamed or misspelled, requesting the url causes Startup.cs to be called (again) and somehow this is throwing exceptions.

<handlers>
  ...
  <remove name="StaticFile" />
  <add name="Owin" verb="" path="/media" type="Microsoft.Owin.Host.SystemWeb.OwinHttpHandler, Microsoft.Owin.Host.SystemWeb"/>
  <add name="ExtensionlessUrlHandler-Integrated-4.0" path="*." verb="*" type="System.Web.Handlers.TransferRequestHandler" preCondition="integratedMode,runtimeVersionv4.0" />
</handlers>

Thanks