A mixed-method approach for the empirical evaluation of the issue-based variability modeling

BackgroundVariability management is the fundamental part of software product line engineering, which deals with customization and reuse of artifacts for developing a family of systems. Rationale approaches structure decision-making by managing the tacit-knowledge behind decisions. This paper reports a quasi-experiment for evaluating a rationale enriched collaborative variability management methodology called issue-based variability modeling.ObjectiveWe studied the interaction of stakeholders with issue-based modeling to evaluate its applicability in requirements engineering teams. Furthermore, we evaluated the reuse of rationale while instantiating and changing variability.ApproachWe enriched a quasi-experimental design with a variety of methods found in case study research. A sample of 258 students was employed with data collection and analysis based on a mix of qualitative and quantitative methods. Our study was performed in two phases: the first phase focused on variability identification and instantiation, while the second phase included tasks on variability evolution.ResultsWe obtained strong empirical evidence on reuse patterns for rationale during instantiation and evolution of variability. The tabular representations used by rationale modeling are learnable and usable in teams of diverse backgrounds.

► A study in the area of product line requirements engineering with a sound empirical rigor.
► We explored the impact of modelers’ motivation on the quality of rationale produced and reused.
► We tested our approach in 42 requirements teams with 258 students from various backgrounds.
► For evaluating requirements approaches, we enriched an experimental design with case-based methods.