How do you add a second test for a function when using GNAT test?

Question

The documentation for Gnat test shows how to generate a harness and skeleton unit tests. It creates one unit test for each public function in the target project, but if I add another this gets overwritten when I regenerate the harness and skeleton unit tests.

In the simple example project provided with gnattest, how do I add another test? I've tried:

(1) Adding another Assert line to the test function that is already there. This works but is not good practice; the first test to fail prevents the others from being run.

(2) Adding a function definition to obj/gnattest/harness/gnattest.xml and then regenerating the tests does not work; the xml file is regenerated before it is used to create the test stubs.

(3) Adding the definition to tests/ and harness/ manually by the changes below, which get clobbered by regenerating tests.

Add this to obj/gnattest/tests/simple-test_data-tests.ads:

   procedure Test_Inc2_4f8b9f (Gnattest_T : in out Test);

Add this to obj/gnattest/tests/simple-test_data-tests.adb:

--  begin read only
   procedure Test_Inc2 (Gnattest_T : in out Test);
   procedure Test_Inc2_4f8b9f (Gnattest_T : in out Test) renames Test_Inc2;
--  id:2.2/4f8b9f38b0ce8c74/Inc/1/0/
   procedure Test_Inc2 (Gnattest_T : in out Test) is
   --  simple.ads:7:4:Inc
--  end read only
   begin
     Assert (Inc(2) = 2, "this test should fail");
--  begin read only
   end Test_Inc2;
--  end read only

Add the declaration to line 16 and the create and add_test to the Suite function in obj/gnattest/harness/simple-test_data-tests-suite.adb:

   Case_1_1_Test_Inc2_4f8b9f : aliased Runner_1.Test_Case;

  Runner_1.Create
    (Case_1_1_Test_Inc2_4f8b9f,
     "simple2.ads:7:4:",
     Test_Inc2_4f8b9f'Access);
  Result.Add_Test (Case_1_1_Test_Inc2_4f8b9f'Access);

To recompile, don't use the Makefile, as that will clobber the changes. Instead, run

gprbuild -Pobj/gnattest/harness/test_driver.gpr
./obj/gnattest/harness/test_runner.exe`.

There has got to be a better way.

Answer 1

There's a GNAT-specific pragma or aspect (Ada 2012), if you don't mind modifying the test subject (and losing a bit of portability in the process...)

I'll show the syntax for pragma first, the rest of the examples will use aspect syntax:

function Foo(Bar : in Integer) return Integer;
pragma Test_Case("Test 1", Robustness);

Same example with aspect syntax:

function Foo(Bar : in Integer) return Integer
   with Test_Case => ("Test 1", Robustness);

Or if you like named association:

function Foo(Bar : in Integer) return Integer
   with Test_Case => (Name => "Test 1", Mode => Robustness);

This will generate additional wrapper code as well as a test case, which may seem like clutter, unless you specify Mode => Nominal (see below) or any of the two optional parameters, Requires and Ensures:

function Foo(Bar : in Integer) return Integer
   with Test_Case => (Name => "Test 1", 
                      Mode => Robustness,
                      Requires => Bar < 10,
                      Ensures => Foo'Result > 15);

Requires acts as a test case-specific pre-condition, while Ensures acts as a test case-specific post-condition, which will be checked by the generated wrapper code. If you have existing Pre or Post-conditions on the test subject, the generated code can check these as well, if you specify Mode => Nominal. (they will be ignored when using Mode => Robustness).

function Foo(Bar : in Integer) return Integer
   with Pre => Bar > 5,
        Post => Foo'Result < 10,
        Test_Case => (Name => "Test 1", 
                      Mode => Nominal,
                      Requires => Bar < 10,
                      Ensures => Foo'Result > 15);

And of course, you can add more than one Test_Case:

function Foo(Bar : in Integer) return Integer
   with Test_Case => ("Test 1", Robustness),
        Test_Case => ("High Bar", Robustness),
        Test_Case => ("Low Bar", Robustness),
        Test_Case => ("Candy Bar", Robustness);