mod_mono: Random CompilationException errors

Question

After a server restart I sometimes get random CompilationException errors like this:

CompilationException: CS0006: Metadata file `/tmp/apache-temp-aspnet-0/ca373c84/assembly/shadow/10cad1cb/02cb7ade_1201ab15_00000001/ZedGraph.dll' could not be found -> HttpException: Single file build failed

and you can't access the application. If I restart the server again then the problem disappears.

The file that triggers the error can be one of my own dll's or a dependency, like mysql.dll for instance.

I am using mono 2.10.2, CentOS release 5.7 (Final), Apache and mod_mono on a Linode VPS.

Any idea of what can be the cause?

Edit: Since my troubles started when I moved to Linode and they use XEN, I have found the option --with-xen_opt=yes that looks promissing: http://mono-project.com/Advanced_Mono_Compile_Options

Answer 1

Sounds like a bug (race condition?) with the shadow copying and compilation. There have been others found and fixed before... see http://grendello.blogspot.com/2007/08/new-modmono-and-xsp-developments.html

And this is a report of a similar problem in 2006 http://lists.ximian.com/pipermail/mono-list/2006-March/030970.html

Rather than rebooting the system, you can try restarting apache. Note, I have found it can help to stop apache and then start, rather than a restart.

You can also enable the mod_mono control panel and restart the application from there. Add this in apache config.

<Location /monocp>
  SetHandler mono-ctrl
  Order deny,allow
  Deny from all
  Allow from 127.0.0.1 <YOURIP>
</Location>

To automate this process take a look at Monit.

Answer 2

FWIW, I had the exact same issue on a dedicated server with no virtualization (8 core i7, x86 Ubuntu Server Edition), so I do not believe this to be a Xen-induced issue. For either the OP (SCL) or anyone else with this issue, I have documented my work around in another SO post here, as well as having filed a bug report with Mono/Xamarin.

It does seem to be a race condition, but not related to virtualization.

Answer 3

It seems that the problems where caused by XEN virtualization and mono precompiled binaries. I compiled mono with the option --with-xen_opt=yes but I wasn't able to compile XSP so I couldn't test if that solved it for sure but I moved exactly the same application to a dedicated server and after 2 million requests I didn't experience any error.

Update: Check Mahmoud answer