SIT: Sampling-based interactive testing for self-adaptive apps

• We propose an interactive application model for understanding interactions between an application and its running environment.
• We propose a test generation technique for systematically sampling an application’s input space in a light-weight way.
• We conduct experiments to evaluate our approach with real-world self-adaptive applications.

Self-adaptive applications (“apps” for short) are useful but error-prone. This stems from developers’ inadequate consideration of environmental dynamics and uncertainty. Two features of self-adaptive apps, infinite reaction loop and uncertain interaction, bring additional challenges to software testing and make existing approaches ineffective. In this article, we propose a novel approach SIT (Sample-based Interactive Testing) to testing self-adaptive apps effectively and in a light-weight way. Our key insight is that a self-adaptive app’s input space can be systematically split, adaptively explored, and mapped to the testing of the app’s different behavior. This is achieved by our approach’s two components, an interactive app model and a test generation technique. The former captures characteristics of interactions between an app and its environment, and the latter uses adaptive sampling to explore an app’s input space and test its behavior. We experimentally evaluated our approach with real-world self-adaptive apps. The experimental results reported that our SIT improved the bug detection by 22.4–42.2%, but with a smaller time cost. Besides, SIT is also scalable with our tailored optimization techniques.