Identification of significant effects from an experimental screening design in the absence of effect sparsity

This paper describes an attempt to derive a new methodology to determine the significance of effects estimated from a screening design, starting from the algorithm of Dong but overcoming its drawbacks. Especially in situations where effect sparsity does not occur and the number of significant effects approaches 50%, the currently often applied algorithm of Dong leads to many important effects incorrectly considered non-significant, i.e. to false negative results. For these situations, a new methodology is recommended. Based on the algorithm of Dong, several alternative approaches were explored and compared. From all approaches, the one using the 75% lowest absolute factor effects to calculate the initial error estimation s0, i.e. s0 = 1.5 × median|E75%|, resulted in the highest number of correct decisions on effects significance. After its definition, the new methodology was tested on a bioanalysis application data set. This study confirmed the earlier conclusions on literature and semisimulated data. The new methodology is especially interesting to be applied in minimal screening designs, for which other error estimates (e.g. based on interaction or dummy effects) cannot be applied.