The power of the sign test given uncertainty in the proportion of tied observations

BackgroundThe sign test is a well-known non-parametric approach for testing whether one of two conditions is preferable to another. In medicine, this method may be used when one is interested in testing in the context of a clinical trial whether either of the two treatments that are provided to study subjects is favored over the other. When neither treatment outperforms the other within a given individual, a “tie” is said to have occurred. When planning such a trial and estimating statistical power and/or sample size, one should consider the probability of a tie occurring (PT). This paper quantifies the degree to which uncertainty in PT affects a study's statistical power.MethodsBinomial theory was used to calculate power given varying levels of uncertainty and varying distributional forms (i.e. beta, uniform) for PT.ResultsAcross a range of prior distributions for PT, power was reduced (i.e. < 80%) for 46 (71.9%) of 64 experimental conditions, with large reductions (i.e. power < 70%) for 10 (15.6%) of them.ConclusionsWhen designing a clinical trial that will incorporate the sign test to compare 2 conditions, ignoring potential variation in the probability of a tie occurring will tend to result in an underpowered study. These findings have implications to the design of any clinical trial for which assumptions are made in calculating an appropriate sample size.