Comparing distributions by multiple testing across quantiles or CDF values

We first show that one-sample and two-sample Kolmogorov-Smirnov tests may be interpreted as multiple testing procedures, nonparametrically testing equality at each point in the distribution with strong control of the finite-sample familywise error rate. Second, we provide an alternative procedure that distributes power across the distribution more evenly than the Kolmogorov-Smirnov test, which suffers low sensitivity to tail deviations. Third, we provide a formula for near-instant one-sample computation. Fourth, we improve power with stepdown and pre-test procedures. Finally, we extend our results to conditional distributions and regression discontinuity designs. Simulations, empirical examples, and code are provided.