Exploring quality measures for the evaluation of feature models: a case study

Evaluating the quality of a feature model is essential to ensure that errors in the early stages do not spread throughout the Software Product Line (SPL). One way to evaluate the feature model is to use measures that could be associated with the feature model quality characteristics and their quality attributes. In this paper, we aim at investigating how measures can be applied to the quality assessment of SPL feature models. We performed an exploratory case study using the COfFEE maintainability measures catalog and the S.P.L.O.T. feature models repository. In order to support this case study, we built a dataset (denoted by MAcchiATO) containing the values of 32 measures from COfFEE for 218 software feature models, extracted from S.P.L.O.T. This research approach allowed us to explore three different data analysis techniques. First, we applied the Spearman's rank correlation coefficient in order to identify relationships between the measures. This analysis showed that not all 32 measures in COfFEE are necessary to reveal the quality of a feature model and just 15 measures could be used. Next, the 32 measures in COfFEE were grouped by applying the Principal Component Analysis and a set of 9 new grouped measures were defined. Finally, we used the Tolerance Interval technique to define statistical thresholds for these 9 new grouped measures. So, our findings suggest that measures can be effectively used to support the quality evaluation of SPL feature models.