Bayesian alternatives to null-hypothesis significance testing for repeated-measures designs

We present a mathematical derivation that establishes the validity of a proposed adaptation to repeated-measures designs of Wagenmakers' (2007) Bayesian information criterion (BIC) method for estimating Bayes factors. We also introduce an improved definition of the penalty in this BIC approximation that accommodates the repeated-measures correlation through an effective sample size based on the Fisher Information. Monte Carlo simulations of repeated-measures data were used to compare the BIC method to two Bayesian procedures for analysis of variance (ANOVA) designs and to the standard null-hypothesis significance testing (NHST) approach. When no effects of the independent variable were present in the populations and a reasonable sample size was used, the Bayesian methods consistently yielded posterior probabilities clearly favoring the null model. We discuss two different approaches to comparing the outcome of the Bayesian analyses with NHST results when an effect is present. In general, a direct comparison between NHST p values and Bayesian posterior probabilities indicates that the latter is somewhat conservative when effect size is small. We also derive a closed-form expression for approximating the posterior probability distributions for condition means in one-factor repeated-measures designs and present an R routine for computing these distributions and the posterior probability of H0 that requires as input nothing more than values from a standard ANOVA.