Is there a way to force JUnit to fail on ANY unchecked exception, even if swallowed

Question

I am using JUnit to write some higher level tests for legacy code that does not have unit tests.

Much of this code "swallows" a variety of unchecked exceptions like NullPointerExceptions (e.g., by just printing stack trace and returning null). Therefore the unit test can pass even through there is a cascade of disasters at various points in the lower level code.

Is there any way to have a test fail on the first unchecked exception even if they are swallowed?

The only alternative I can think of is to write a custom JUnit wrapper that redirects System.err and then analyzes the output for exceptions.

Answer 1

If you execute the tests in your IDE's debugger you can configure the IDE to break when an exception is thrown.

Answer 2

In lack of a concrete solution, my answer's pretty general:

Such code smells (e.g. swalling exceptions) are best cleaned up step by step (class by class) when you encounter them while bugfixing the legacy system. Code quality tools (e.g. Findbugs, PMD, Checkstyle or even Sonar Quality Server) help you find those things.

A way to "catch" swallowed exceptions automatically is to use AspectJ compiler. You can declare aspects to generate a compile-time error in your IDE when certain code conventions are violated. Alternatively, you can weave at runtime the classes under test and let AspectJ rethrow such exceptions, so they can be recorded by the JUnit runner.

Answer 3

It could be possible to mock the Exceptions you want to catch with a mocking framework that modifies the bytecode, or with an AOP framework. I imagine you could modify the constructor to throw a more fatal Exception, or to set some flag in your test code.

Answer 4

I believe Exception is just a standard class in the SDK libraries.

If you extracted it, modified it and put it somewhere on your classpath before the SDK, I think it should replace the one in the SDK (If not you could put your "new" exception back into the SDK jar)

Anyway, your new exception could set a static value that can be read by the testing framework.

May not be the most elegant solution but it doesn't require any "Magic"

Answer 5

I would try to use AOP to throw failures. Something like this should work (note, I haven't tested this, and you obviouslly need to have AspectJ Setup to use AOP annotations)

public class ClassUnderTest
{
   private static Boolean exceptionThrown;


   @Before
   public void resetExceptionFlag()
   {
      ClassUnderTest.exceptionThrown = false;
   }

   @Test
   public void myTestMethod()
   {

      //....
      // My Test Exception Code
      //....

      assertFalse(ClassUnderTest.exceptionThrown);
   }

   @Aspect
   static class TestAspects
   {
      @Pointcut("handler(Exception)")
      public void intereceptAllExceptions(){}

      //This is Fully Qualified because of the conflict with the junit Before annotation above
      @org.aspectj.lang.annotation.Before("intereceptAllExceptions()")
      public void flagExceptionThrown()
      {
         ClassUnderTest.exceptionThrown = true;

  }

} }

Answer 6

Perhaps you can paste the code that you are trying to test. Mostly likely you would have to use some of "remapper" testing frameworks such as powermock or jmockit to manipulate the classloader manipulation in JVM. But a sample of class under test would help determine the approach required.