Evolutionary functional testing

The development and testing of software-based systems is an essential activity for the automotive industry. The 50–70 software-based systems with different complexities and developed by various suppliers are installed in today's premium vehicles, communicating with each other via different bus systems. The integration and testing of systems of this complexity is a very challenging task. The aim of testing is to detect faults in the systems under test and to convey confidence in the correct functioning of the systems if no faults are found during comprehensive testing. Faults not found in the different testing phases could have significant consequences that range from customer dissatisfaction to damage of physical property or, in safety-relevant areas, even to the endangering of human lives. Therefore, the thorough testing of developed systems is essential. Evolutionary testing tries to improve the effectiveness and efficiency of the testing process by transforming testing objectives into search problems, and applying evolutionary computation in order to solve them. The most important class of testing methods is functional testing. However, functional testing is difficult to automate by evolutionary testing. This work will describe how evolutionary testing could be applied to automate functional testing in general and the testing of complex automotive systems in particular. It presents two case studies and shows how evolutionary testing is effective at finding faults in the functional behaviour of these systems. In addition, a quantitative comparison with manual and random test case selection is done for one application.