The variance of screening and supersaturated design results as a measure for method robustness

Screening designs are factorial designs to evaluate the importance of factors in a number of experiments that is at least one higher than the number of factors examined. Supersaturated designs are factorial designs with more factors than experiments. These designs do not allow a correct estimation of the factor effects due to a confounding of main effects. Therefore, it is evaluated whether, in robustness testing, the variance of a response can be used as a measure for the robustness of a method. A number of potential reference criteria (reference variances estimating reproducibility and limit values) also are evaluated for their applicability to decide whether the examined factors cause non-robustness. Finally, it was also examined which conclusions one statistically can draw from comparing the variances from the design experiments with the reference criteria. Two approaches are considered for the reference variances: a classical F-test and interval hypothesis testing. The use of some limit values was also discussed. It was found that the variance of a response could be used as a measure for robustness, but statistically this variance could not be interpreted in an acceptable way. Either a large probability to accept a non-robust or to reject a robust method occurs due to the small number of degrees of freedom to examine a given number of factors, especially when applying supersaturated designs.