We recently upgraded our server O/S from Win2012 to Win2019. While on 2012, in IIS, we had configured a Virtual Directory called PDFReader. The physical path was a filestream directory which contains a series of PDFs, which is essentially just a UNC share.
The directory was being called by an SSRS report & its sole purpose was to call the PDF from filestream and render it in SSRS.
We used a simple virtual directory when we were on Windows 2012 which worked fine. But we noticed that the app pool kept crashing on 2019. So that led to me convert it to an application because that way, I could assign an individual app pool to it & manage the rapid fail protection settings as a band-aid for now until we could identify the underlying issue. That has been getting us by.
I tried a few things such as changing the application pool identity to LocalSystem, then a privileged domain user w/SQL Filestream access. I also tried changing the .NET CLR Version to "No Managed Code" and the pipeline from Integrated to Classic. The latter seemed to slow the failures slightly, but nevertheless the problem persists.
Here's the stacktrace which we consistently see every few minutes as users frequently pull up PDFs.
Faulting application name: w3wp.exe, version: 10.0.17763.1, time stamp: 0xa7ccada2 Faulting module name: nativerd.dll, version: 10.0.17763.1, time stamp: 0x84a81d7a Exception code: 0xc0000005 Fault offset: 0x0000a8f0 Faulting process id: 0x25cc Faulting application start time: 0x01d921151b65f242 Faulting application path: C:\Windows\SysWOW64\inetsrv\w3wp.exe Faulting module path: C:\Windows\System32\inetsrv\nativerd.dll Report Id: 88535cc3-8716-40f7-bfab-1f53f9c77b92 Faulting package full name: Faulting package-relative application ID:
Fault bucket 1192683016347496699, type 1 Event Name: APPCRASH Response: Not available Cab Id: 0
Problem signature: P1: w3wp.exe P2: 10.0.17763.1 P3: a7ccada2 P4: nativerd.dll P5: 10.0.17763.1 P6: 84a81d7a P7: c0000005 P8: 0000a8f0 P9: P10:
There are a number of PDFs that are called which one might consider large, over 20MB with a handful of them over 100mb.
As for the user experience, since I implemented the rapid fail protection, the user doesn't run into the 503 errors anymore, requiring me to start the stopped app pool. Occasionally, when the worker process dies & hands off to the next one spawned, it's a minor delay.
Any help or insight would be appreciated.
Thanks, Mike
- Converted virtual directory to application.
- Created/assigned individual app pool to application.
- Changed rapid fail protection settings to allow for 25 failures in 10 minutes instead of the 5/5 out of the box.
- Changed application pool identity to LocalSystem... Problem persisted. So I changed to a privileged domain user/local admin with proper SQL filestream rights, same behavior.
- Ensured nativerd.dll was present in system directory, no discernible permissions issues.
- Looked into MaxWebConfigFileSizeInKB registry key associated with nativerd.dll per a Microsoft KB article, but I don't see that key present in the registry.
- Ran DebugDiag analysis with this summary result:
In w3wp__PDFReader__PID__160__Date__06_15_2022__Time_08_39_35AM__249__Second_Chance_Exception_C0000005.dmp the assembly instruction at nativerd!INDEXED_ARRAY_LIST::FindEntry+18 in c:\Windows\System32\inetsrv\nativerd.dll from Microsoft Corporation has caused an access violation exception (0xC0000005) when trying to read from memory location 0x00000038 on thread 6