On strategies for testing software product lines: A systematic literature review

ContextTesting plays an important role in the quality assurance process for software product line engineering. There are many opportunities for economies of scope and scale in the testing activities, but techniques that can take advantage of these opportunities are still needed.ObjectiveThe objective of this study is to identify testing strategies that have the potential to achieve these economies, and to provide a synthesis of available research on SPL testing strategies, to be applied towards reaching higher defect detection rates and reduced quality assurance effort.MethodWe performed a literature review of two hundred seventy-six studies published from the year 1998 up to the 1st1st semester of 2013. We used several filters to focus the review on the most relevant studies and we give detailed analyses of the core set of studies.ResultsThe analysis of the reported strategies comprised two fundamental aspects for software product line testing: the selection of products for testing, and the actual test of products. Our findings indicate that the literature offers a large number of techniques to cope with such aspects. However, there is a lack of reports on realistic industrial experiences, which limits the inferences that can be drawn.ConclusionThis study showed a number of leveraged strategies that can support both the selection of products, and the actual testing of products. Future research should also benefit from the problems and advantages identified in this study.