Mutation Score vs Test Code Coverage

January 13, 2019 ยท View on GitHub

A mutation score differs from the test code coverage metric generated by many tools, and it's valuable to consider the two metrics alongside one another.

Test code coverage is only a metric of how much of your application code is executed in a test suite. It's possible to have 100% code coverage and a mutation score of 0. This would be a test suite that only executes code but never makes any assertions against the code.

It's not possible to have a mutation score > 0 and to have 0% code coverage. This would mean that you don't execute any of your application code in your test suite but your test suite somehow caught errors.

A high mutation score, with a high amount of code coverage coverage, indicates a very rigorous test suite; a lot of your application code is executed in your test suite, and the assertions that you're using in your test suite generally catch any introduced mutants.

A high mutation score, with a low amount of code coverage, indicates that a lot of your application code is not tested, but that which is tested has robust tests validating the logic.

A low mutation score, with a high amount of code coverage, indicates that a lot of your application code is being executed in a test, but that the tests you have written are not of a high quality.