Why does my AWS CodeBuild test report result in an 'Incomplete' status?

Question

I am trying to understand why my JUnit XML report results in an Incomplete status on AWS CodeBuild.

The XML is produced by Kaocha, a Clojure test runner, through its kaocha-junit-xml plugin.

At the end of my test run, the XML is generated and then processed in the UPLOAD_ARTIFACTS phase which is where it does a calculation and that results in:

error processing report: [InvalidInputException: Test summary: status count cannot be negative]]

I do have multiple assertions per test, and thus there may be more than 1 failure per test. To verify that I'm not having a buggy JUnit XML file, I have installed Jenkins and ran a couple of tests, which works and it does not end in an Incomplete Report status.

Note that the Test Run status is Failed, and only the Report status is Incomplete.

Answer 1

Please review your testsuite property in the JUnit XML, e.g.

tests="1" failures="1" errors="1"

Here, the failures and errors are treated differently and this makes the errors = 2 (1+1) which is less than the number of tests (= 1) causing the negative status count (1-2).

I am not sure about the JUnit format, but if you can tweak it and make sure either failure or error is populated (not both), then this error ("status count cannot be negative") will not appear.