To adjust or not adjust: Nonparametric effect sizes, confidence intervals, and real-world meaning

ObjectivesThe main objectives of this article are to: (a) investigate if there are any meaningful differences between adjusted and unadjusted effect sizes (b) compare the outcomes from parametric and non-parametric effect sizes to determine if the potential differences might influence the interpretation of results, (c) discuss the importance of reporting confidence intervals in research, and discuss how to interpret effect sizes in terms of practical real-world meaning.DesignReview.MethodA review of how to estimate and interpret various effect sizes was conducted. Hypothetical examples were then used to exemplify the issues stated in the objectives.ResultsThe results from the hypothetical research designs showed that: (a) there is a substantial difference between adjusted and non-adjusted effect sizes especially in studies with small sample sizes, and (b) there are differences in outcomes between the parametric and non-parametric effect size formulas that may affect interpretations of results.ConclusionsThe different hypothetical examples in this article clearly demonstrate the importance of treating data in ways that minimize potential biases and the central issues of how to discuss the meaningfulness of effect sizes in research.

► A number of statistics could be used in order to interpret effect sizes.
► Calculation of non-parametric effect sizes should be used for non-normal data.
► We suggest authors to adjust effect sizes especially when sample sizes are small.