Confidence intervals for an ordinal effect size measure based on partially validated series

An ordinal effect size measure is used to assess whether one variable is stochastically larger than the other; therefore, this measure is a useful means by which to describe the difference between two ordinal categorical distributions. In practical analysis, it is desirable to obtain data only by a gold standard test, but such tests are often limited due to high costs or ethical considerations, especially in medical research applications. However, misclassification can arise when a cheaper, faster, and/or non-invasive, but fallible test is used to collect data. The use of partially validated data obtained by double sampling has become a popular compromise between these two approaches. In this study, we develop twelve estimators of the confidence interval (CI) for an ordinal effect size measure based on partially validated data. The performance of the proposed CIs are evaluated by simulation studies in terms of the empirical coverage probability, the empirical coverage width, and the ratio of the mesial non-coverage probability and non-coverage probability. Simulation results show that the Wald CI on logit scale, the Bootstrap percentile CI and the Bias-corrected Bootstrap normal CI have outstanding performance even in small sample designs. When sample sizes are moderate, all CIs except the Wald, Bias-corrected Bootstrap percentile and logit-transformation-based Bootstrap percentile-t CIs demonstrate good coverage properties. Moreover, all CIs perform well when sample sizes are large. All methods are illustrated by analyzing a real data set from a research study of highway safety on automobile accidents.