Mitigating the effects of equivalent mutants with mutant classification strategies

• A mutation analysis process based on mutant classification.
• Two dynamic strategies aiming at reducing the effects of equivalent mutants.
• An empirical evaluation of the effectiveness and efficiency of mutant classification schemes.
• A cost-benefit comparison of the mutant classification schemes with the traditional mutation testing approach.

Mutation Testing has been shown to be a powerful technique in detecting software faults. Despite this advantage, in practice there is a need to deal with the equivalent mutants' problem. Automatically detecting equivalent mutants is an undecidable problem. Therefore, identifying equivalent mutants is cumbersome since it requires manual analysis, resulting in unbearable testing cost. To overcome this difficulty, researchers suggested the use of mutant classification, an approach that aims at isolating equivalent mutants automatically. From this perspective, the present paper establishes and empirically assesses possible mutant classification strategies. A conducted study reveals that mutant classification isolates equivalent mutants effectively when low quality test suites are used. However, it turns out that as the test suites evolve, the benefit of this practice is reduced. Thus, mutant classification is only fruitful in improving test suites of low quality and only up to a certain limit. To this end, empirical results show that the proposed strategies provide a cost-effective solution when they consider a small number of live mutants, i.e., 10–12. At this point they kill 92% of all the killable mutants.