Calculating Test Strength from the XML output, what does 'detected' represent?

Question

I am attempting to calculate the Test Strength generated by the HTML output from the data within the XML output of PIT however the figure I calculate doesn't align with the Test Strength from the HTML report.

I use the equation (mutationsKilled / (totalMutations - mutationsNotCovered) * 100), where:

• mutationsKilled was generated by counting the entries where status = "KILLED"
• totalMutations was generated by counting the total entries
• mutationsNotCovered was generated by counting the entries where detected = "false"

However, it's the last of these that I suspect could be the problem, as when looking back at the raw XML output I find that detected metric may not necessarily indicate whether the mutation was covered or not.

Is there a tried and tested way of doing this? Also may be worth mentioning I'm running PIT 1.9.4 and also the incremental analysis mode, if this has any bearing on the issue.

Answer 1

After experimenting and intentionally reducing my test coverage to produce uncovered mutants, I found that there is a possible status, NO_COVERAGE. By adjusting the above equation to generate mutationsNotCovered by counting the entries where status = "NO_COVERAGE", the correct statistics are produced.