Test diversity

This paper describes a novel method for measuring the degree to which a set of test cases executes a given program in diverse ways with respect to the two fundamental programming concepts: control and data. Test diversity is a method for measuring the variety of software control flow and data flow, comprising of four new measures: conditional diversity, data diversity, standard deviation of diversity, and test orthogonality. These closely related measures could be used to evaluate the test effectiveness and the test-effort distribution of a test suite.The Diversity Analyzer is a novel industrial-strength testing tool that can currently perform diversity analysis on software written in C/C++/C#/VB/Java in Windows and .NET environments. The Diversity Analyzer is used to evaluate the fault-detection effectiveness of Test Diversity on various types of industrial projects.