CMake and CTest: automatically run test's dependencies

Question

In my CMakeLists.txt I have something like this:

 set_property(TEST testX APPEND PROPERTY DEPENDS testY)
 set_property(TEST testX APPEND PROPERTY DEPENDS testZ)

And I want some way to run testX and all its dependencies automatically. Something like:

ctest <options> testX

and as a result, CTest will run textY, testZ and testX. Is there any way to do this?

Or, if it is impossible now, is there any way to extract information on the dependencies from the CMake build directory by script?

Answer 1

There's no built-in way of doing this as far as I know.

The best way I can think of to achieve your goal is to use the LABELS property on the tests. You can retrieve the list of dependencies using get_property or get_test_property and apply the same label to testX and each of its dependent tests:

get_test_property(testX DEPENDS DependenciesOfTestX)
set_tests_properties(testX ${DependenciesOfTestX} PROPERTIES LABELS LabelX)

Then you can tell CTest to only run tests with that label:

ctest -L LabelX

Answer 2

Support for test fixtures was added in CMake 3.7 and they do exactly what you want. Your particular scenario would be implemented like this:

set_property(TEST testX PROPERTY FIXTURES_REQUIRED Foo)
set_property(TEST testY PROPERTY FIXTURES_SETUP Foo)
set_property(TEST testZ PROPERTY FIXTURES_SETUP Foo)

You can then ask ctest to run just testX and it will automatically add testY and testZ to the set of tests to be executed:

ctest -R testX

It will also ensure that testX will only run after testY and testZ have been passed. If either of testY or testZ fails, testX will be skipped. New options -FS, -FC and -FA were also added to ctest in CMake 3.9 which allow automatic adding fixture setup/cleanup tests to be controlled at the ctest command line. For example, to temporarily skip adding testY to the test set but still automatically add testZ, one could do this:

ctest -R testX -FS testY

The fixtures properties are described in the CMake docs and the following article covers the fixtures feature more comprehensively:

https://crascit.com/2016/10/18/test-fixtures-with-cmake-ctest/