Boost Test run function on failure

Question

I have many tests which depend on stochastic procedures. Rarely, the tests fail, as you basically get unlucky with the random numbers.

However, sometimes the failure is actually a bug, and I'd like to reproduce it. I have a global random generator that seeds everything else. I'd like for Boost Test to output its original seed in case any test case fails.

The closest thing I found would be to output the original seed in a Fixture, but it seems I'd have to print it all the time, rather than just on failure. Is there a simple, concise way to print a statement in case any test fails?

Answer 1

I think I managed to find a solution, where I use a Visitor (courtesy of this answer) to figure out whether all test cases have passed, and if not print my needed info.

struct SeedPrinter {
    struct AllPassVisitor : boost::unit_test::test_tree_visitor {
        bool passed = true;

        void visit( boost::unit_test::test_case const& test ) {
            passed &= boost::unit_test::results_collector.results(test.p_id).passed();
        }
    };

    ~SeedPrinter() {
        namespace ut = boost::unit_test;

        // Check whether we have passed all tests.
        AllPassVisitor v;
        ut::traverse_test_tree(ut::framework::master_test_suite(), v);

        // If not, print the global seed to reproduce errors.
        if (!v.passed)
            BOOST_CHECK_MESSAGE(false, "PRINT MY INFO");
    }
};

BOOST_TEST_GLOBAL_FIXTURE(SeedPrinter);

Note that to print I use BOOST_CHECK_MESSAGE because BOOST_TEST_MESSAGE does not show up in the output unless --log_level=message is specified.

If anybody can figure out a better solution I'm all ears :)

Answer 2

Enclose your test in a BOOST_TEST_CONTEXT. For example:

BOOST_AUTO_TEST_CASE( TestSeed )
{
    int seed = getSeed();
    BOOST_TEST_CONTEXT( "seed value is " << seed )
    {
        // BOOST_CHECK() statements, etc., that sometimes fail
    }
}

This will output the seed value if and when the test fails.