Best practices for repeated measures ANOVAs of ERP data: Reference, regional channels, and robust ANOVAs

- Effects of reference channel on ANOVA depended on its noise level as well as its condition effect, with average reference benefitting from overall lower noise level.
- Regional channels (averaging over adjoining electrodes) generally improved noise levels but diluted effects.
- An omnibus regional channels ANOVA was not as effective as testing peak channels.
- Robust ANOVAs involve a stochastic element that should be managed and a procedure is suggested.
- In the case of this dataset, robust ANOVA seemed to reduce p-values but the pattern of effects were more in line with expectations.

Analysis of variance (ANOVA) is a fundamental procedure for event-related potential (ERP) research and yet there is very little guidance for best practices. It is important for the field to develop evidence-based best practices: 1) to minimize the Type II error rate by maximizing statistical power, 2) to minimize the Type I error rate by reducing the latitude for varying procedures, and 3) to identify areas for further methodological improvements. While generic treatments of ANOVA methodology are available, ERP datasets have many unique characteristics that must be considered. In the present report, a novelty oddball dataset was utilized as a test case to determine whether three aspects of ANOVA procedures as applied to ERPs make a real-world difference: the effects of reference site, regional channels, and robust ANOVAs. Recommendations are provided for best practices in each of these areas.