What is a mutation test?
By definition, a “mutation” is:
A permanent change in an organism or the changed organism itself.
Cambridge Dictionary
So by applying this concept to our code, a mutant is a version of the code in which a small change has been made deterministically, and, of course, we expect the test fails against this.
Let’s see an example:
if(someString == null)It will be changed to:
if(someString != null)After this small change (a mutation), we should rerun the tests, and if you wrote all the tests correctly, at least one should fail. By doing this, we kill the mutant we just created. But if all the test cases were successful, the mutant survived.
This is a problem! If someone introduces this change unconsciously, the test cases will not catch that, and we will have a bug in our code!
Advantages of Mutation test
A mutation test is a powerful technique for improving the quality and effectiveness of software testing, and some of the main advantages include the following:
Detection of weak spots in the test suite:
Mutation testing can identify areas of the code not sufficiently covered by the test suite. By introducing many mutations into the code, mutation testing can reveal which areas of the code are most challenging to test and where additional test cases may be needed.Validation of the test suite:
Mutation testing ensures that the test suite effectively catches real faults in the code. By comparing the results of running the test suite against the original and mutated codes, mutation testing can objectively measure the test suite’s quality.Improvement of code quality:
Mutation testing can help identify weaknesses in the code itself, such as redundant or poorly written code. By highlighting areas of the code prone to faults, mutation testing can encourage developers to improve the quality of the code they write.Prevention of regression bugs:
Mutation testing can help ensure that code changes do not introduce new bugs into the system. By running the test suite against the mutated code, mutation testing can reveal if any changes made to the code have unintentionally caused the test suite to fail.
How to perform mutation testing
We just saw what a mutation test is and its benefits to ensure that our test cases have been written correctly and, by doing so, prevent the introduction of new bugs in our code.
An essential tool has been used to ensure that our unit test has covered all our code (or some percentage of them). So we wrote our test cases and put a gate on the pipeline to guarantee the code rate was covered, but will 100% of coverage prevent some changes breaks the code? Let’s write some code:
public class TimeConversion {
public String convert(String s) {
int givenHour = Integer.parseInt(s.substring(0, 2));
var suffix = s.substring(8);
int convertedHour;
if (suffix.equalsIgnoreCase("PM") && givenHour < 12) {
convertedHour = givenHour + 12;
} else if (suffix.equals("AM") && givenHour >= 12) {
convertedHour = givenHour - 12;
} else {
convertedHour = givenHour;
}
return String.format("%02d", convertedHour) + s.substring(2, 8);
}
}So, we need a test case for the previous class:
import org.junit.Assert;
import org.junit.Test;
public class TimeConversionTest {
@Test
public void testTimeConversion() {
TimeConversion timeConversion = new TimeConversion();
Assert.assertEquals("23:00:00", timeConversion.convert("11:00:00PM"));
Assert.assertEquals("00:00:00", timeConversion.convert("12:00:00AM"));
}
}Ok, this is a simple method with a unit test, and as lucky as we are, the test covers all lines. Look:
