Principles: When there should be no difference – how to fail to reject the null hypothesis

It is common to perform experiments in which a ‘success’ is claimed when the null hypothesis is discarded. However, there is a category of experiment that has become important in which a success is when the null hypothesis is not rejected. Failing to discard the null hypothesis is different from proving it to be valid, a distinction that is particularly important in experiments in which any inadequacy of experimental design or implementation enhances the likelihood of a success. The appropriate analysis of such experiments tests for evidence of the validity of the null hypothesis rather than simply failing to find evidence against it.