Towards the practical mutation testing of AspectJ programs

Mutation testing is a test selection criterion that relies on the assumption that test cases which can reveal artificial faults in the software are also good to reveal the real ones. It helps to expose faults which would go otherwise unnoticed. This criterion has been shown to be a promising means to deal with testing-related specificities of contemporary programming techniques such as Aspect-Oriented Programming. However, to date the few initiatives for customising mutation testing for aspect-oriented (AO) programs show either limited coverage with respect to the range of simulated faults, or a need for both adequate tool support and proper evaluation in regard to properties like application cost and effectiveness. This article tackles these limitations by describing a comprehensive mutation-based testing approach for programs written in AspectJ, which represents the most investigated AO programming language to date. The approach encompasses the definition of a set of mutation operators for AspectJ-specific constructs and the implementation of a tool that automates the approach. The results of a preliminary evaluation study show that the mutation operators are able to simulate faults that may not be revealed by pre-existing, non-mutation-based test suites. The results also suggest that the approach seems not to overwhelm the testers and hence represents a step towards the practical fault-based testing of AspectJ-like programs.