Comparing model-based and dynamic event-extraction based GUI testing techniques: An empirical study

• We compare model-based and dynamic event-extraction based GUI testing techniques.
• We present a exible GUI testing framework that enables comparison of techniques.
• We show that nonexecutable events weaken effectiveness and increase costs of testing.
• We show that high statement coverage does not imply high numbers of faults detected.
• We show that test case length affects fault-detection and coverage.

Graphical user interfaces are pervasive in modern software systems, and to ensure their quality it is important to test them. Two primary classes of automated GUI testing approaches, those based on static models and those based on dynamic event-extraction, present tradeoffs in cost and effectiveness. For example, static model-based GUI testing techniques can create test cases that contain nonexecutable events, whereas dynamic event-extraction based GUI testing techniques can create larger numbers of duplicate test cases. To better understand the effects of these tradeoffs, we created a GUI testing framework that facilitates fair comparison of different GUI testing techniques, and we conducted a controlled experiment comparing representative versions of static-model based and dynamic event-extraction based testing techniques on several GUI-based Java applications. Our study reveals several cost and effectiveness tradeoffs between the techniques, with implications for research and practice.