Your project has unit tests. Your project has 100% coverage. But what if your tests are susceptible to reporting false positives? We write unit tests to build confidence that our code performs as expected, but how can you be confident that your tests are making the right assertions against your code?
Enter mutation testing: a way to supplement your code coverage that confirms your tests fail when breaking code changes are made. In this video, let’s learn how these mutation tests work.
Ready to Add Mutation Testing to Your Project?
Here are some things to keep in mind as you get started:
1. Setup is easy and requires very few changes
Since mutation testing works using existing unit tests, adding this to your testing strategy requires little overhead. This is different from adding additional testing strategies, like end to end or visual regression testing because it doesn’t require writing additional tests—unless, of course, mutants are detected. If they are, it might be a flag that you need to write better unit tests.
2. It can help you write better code
Mutation testing can be valuable in detecting hidden faults in your source code that might otherwise go unnoticed—like syntactic errors that cause bugs. Error-revealing mutations help isolate logical errors in your source code when mutations are detected even with adequate assertions in your unit tests.
3. Mutation testing is a partner to coverage, not a replacement
Mutation testing doesn’t completely replace coverage because sometimes it’s all about speed. While mutation testing is thorough, it’s not always fast. Since it runs through your test suite multiple times, the entire process can take minutes—or even hours—to finish running depending on the complexity of your codebase. When writing new tests or refactoring your source code, speed is important! Code coverage is an important tool because it produces fast feedback during certain stages of test writing that allows you to quickly determine where coverage is lacking. Mutation testing should be used after your coverage is satisfactory and you want to make sure your tests aren’t susceptible to false positives.