Formulating research policy on expert advice

•National assessments are used to promote research quality throughout the world.•They rely on the advice of experts who are themselves subject to evaluation.•We develop a theoretical model that explains how experts formulate their advice.•We use a unique dataset to investigate the predictions of the model.•We find cognitive biases to dominate expert assessments.

National Research Assessments represent the principal policy used by countries with public university systems to improve the quality of their universities׳ research. They typically employ expert panels of local academics to assess journal quality, and use such assessments to reward publications. We propose a model of how experts assess quality and test this model using data from the recent Australian assessment. Controlling for objective quality, we show that experts׳ cognitive biases dominate their assessments. Consequently, such assessments exaggerate the quality of the experts׳ research and understate that of their colleagues. Our model can explain 95% of the variation in assessments.