Integrated fault detection and test algebra for combinatorial testing in TaaS (Testing-as-a-Service)

Testing-as-a-Service (TaaS) is a software testing service in a cloud that can leverage the computation power provided by the cloud. Specifically, a TaaS can be scaled to large and dynamic workloads, executed in a distributed environment with hundreds of thousands of processors, and these processors may support concurrent and distributed test execution and analysis. This paper proposes a TaaS system based on Adaptive Reasoning (AR) and Test Algebra (TA) for Combinatorial Testing (CT). AR performs testing and identifies faulty interactions, and TA eliminates related configurations from testing and there can be carried out concurrently. By combining these two, it is possible to perform large CT that were not possible before. Specifically, we performed experiments with 250 components with 2.83*1087 6-way interactions with about 21.1×1015 configurations, and this may be the largest CT experimentation as 2014. 98.6% of configurations have been eliminated out of total number of configurations.