Input-output conformance testing for software product lines

- Extending the theory of input out conformance to support featured transition systems.
- Defining notions of refinement on models and test-suites and showing their correspondence.
- Definition notions of residual and spinal test suites to save on test effort.
- Analyzing the relative testing power of residual and spinal test suites and their completeness.
- Analyzing an example regarding the Ceiling Speed Monitoring Function from the European Train Control System.

We extend the theory of input-output conformance (IOCO) testing to accommodate behavioral models of software product lines (SPLs). We present the notions of residual and spinal testing. These notions allow for structuring the test process for SPLs by taking variability into account and extracting separate test suites for common and specific features of an SPL. The introduced notions of residual and spinal test suites allow for focusing on the newly introduced behavior and avoiding unnecessary re-test of the old one. Residual test suites are very conservative in that they require retesting the old behavior that can reach to new behavior. However, spinal test suites more aggressively prune the old tests and only focus on those test sequences that are necessary in reaching the new behavior. We show that residual testing is complete but does not usually lead to much reduction in the test-suite. In contrast, spinal testing is not necessarily complete but does reduce the test-suite. We give sufficient conditions on the implementation to guarantee completeness of spinal testing. Finally, we specify and analyze an example regarding the Ceiling Speed Monitoring Function from the European Train Control System.