Improving the rigor of psychophysiology research

- Reviews how the incentives of academic culture can work against rigor
- Reviews the problems with researcher flexibility
- Reviews the impact of statistical power on the positive predictive value and exaggeration of effects
- Reviews five suggestions about how to improve methodological practices to increase rigor

Psychology as a field is in the midst of what is sometimes called a “crisis” because false findings are prevalent. Although the focus of the methodological and substantive criticisms of psychology has focused on social psychology, psychophysiology research is not without its problems. The author discusses (a) researcher flexibility and its impact on the stability of conclusions and (b) the role power plays in the probability that a finding is true and the precision of estimates. The author uses examples and data from psychophysiological research to illustrate the problems. The author concludes with a discussion of ways to shift the practice of science to improve the reliability of findings. Suggestions for improvement include: increased power through collaboration, improved statistical and methodological training, pre-registration of studies, improved reporting standards, and shifting incentives surrounding hiring and promotion.