Asymptotically valid and exact permutation tests based on two-sample UU-statistics

• The two-sample Wilcoxon test generally fails to control the probability of Type 1 error.
• We provide a general theory whereby one can construct a permutation test of a parameter θ(P,Q)=θ0θ(P,Q)=θ0.
• The new method controls both the asymptotic probability of the Type 1 error and the exactness property.
• The new studentized Wilcoxon test retains all the benefits of the usual Wilcoxon test.
• The results are derived for general two-sample UU-statistics.

The two-sample Wilcoxon test has been widely used in a broad range of scientific research, including economics, due to its good efficiency, robustness against parametric distributional assumptions, and the simplicity with which it can be performed. While the two-sample Wilcoxon test, by virtue of being both a rank and hence a permutation test, controls the exact probability of the Type 1 error under the assumption of identical underlying populations, it in general fails to control the probability of the Type 1 (or Type 3) error, even asymptotically. Despite this fact, the two-sample Wilcoxon test has been misused in many applications. Through examples of misapplications in academic economics journals, we emphasize the need for clarification regarding both what is being tested and what the implicit underlying assumptions are. We provide a general theory whereby one can construct a permutation test of a parameter θ(P,Q)=θ0θ(P,Q)=θ0 which controls the asymptotic probability of the Type 1 error in large samples while retaining the exactness property when the underlying distributions are identical. In addition, the new studentized Wilcoxon test retains all the benefits of the usual Wilcoxon test, such as its asymptotic power properties and the fact that its critical values can be tabled (which we provide in the supplementary appendix). The results are derived for general two-sample UU-statistics. A key ingredient that aids our asymptotic derivations is a useful coupling method.