Assessing a requirements evolution approach: Empirical studies in the air traffic management domain

• We conducted empirical studies in the ATM domain to evaluate the effectiveness of an approach to model requirements change.
• The studies involved participants with different level of knowledge in the approach and in the ATM domain.
• The studies show that the approach is effective in modeling requirements evolution.
• The studies show that domain knowledge and method knowledge have no observable effect on the effectiveness of the approach.

Requirements evolution is still a challenging problem in engineering practices. In this paper, we report the results of the empirical evaluation of a novel approach for modeling and reasoning on evolving requirements. We evaluated the effectiveness of the approach in modeling requirements evolution by means of a series of empirical studies in the air traffic management (ATM) domain. As we also wanted to assess whether the knowledge of the method and/or the application domain influences the effectiveness of the approach, the studies involved researchers, master students and domain experts with different level of knowledge of the approach and of the ATM domain. The participants have applied the approach to a real evolutionary scenario which focuses on the introduction of a new queue management tool, the Arrival MANager (AMAN) and a new network for information sharing (SWIM) connecting the main ATM actors. The results from the studies show that the modeling approach is effective in capturing requirements evolution. In addition, domain knowledge and method knowledge do not have an observable effect on the effectiveness of the approach. Furthermore, the evaluation provided us useful insights on how to improve the modeling approach.