Means and standard deviations, or locations and scales? That is the question!

According to standard experimental practice, researchers randomly assign participants to experimental and control conditions, deeming the experiment “successful” if the means of the two conditions differ in the hypothesized direction. Even for complex experiments, with many conditions, success generally depends on a comparison or contrast of means across conditions. Because the experimental manipulation may change the shape of the distribution, we show that a difference in means, even if large and in the hypothesized direction, does not necessarily indicate the success of the experiment. To make this determination, it also is necessary to compute location statistics. It is possible for means to change but for locations not to change, for means not to change but for locations to change, and for mean differences and location differences to be in opposite directions. Therefore, typical research that depends on differences between means across conditions, cannot be trusted in the absence of location statistics. For similar reasons, nor can standard deviations be trusted without scale statistics. Therefore, we take the radical step of arguing that all researchers who report means and standard deviations, also should be required to report corresponding location and scale statistics.