A theory-based intervention to prevent calibration effects in serial sport performance evaluations

• We assume that serial position effect occur in subjective serial judgments.
• This effect is based on a lack of calibration on the actual performance context.
• The present study examines an intervention to prevent this effect.
• Calibration could be facilitated by an advanced range of performance presentation.

ObjectivesSerial performance evaluations show calibration effects: Judges avoid extreme categories in the beginning (e.g. best or worst) because they need to calibrate an internal judgment scale (Unkelbach et al., 2012). Successful calibration is therefore important for fair and unbiased evaluations. A central prerequisite for successful calibration is knowledge about the performance range. The present study tests whether advance knowledge about the range (best and worst) of performances in a series reduces calibration effects.DesignA 2 × 2 × 2 design was developed with two between subject factors: the knowledge about the performance range (with vs. without) and two different talent tests (specific vs. unspecific). As within subject factor the position of the performances in the series (position 1–10 vs. 11–20) was integrated. The combination of the between subject factors resulted in four experimental conditions.MethodHandball coaches were randomly assigned to one of the conditions. Afterwards twenty performances were evaluated in a randomized order by the coaches.ResultsWithout knowledge about the range, they showed the expected avoidance of extreme categories in the beginning independent of the presented talent test. However, observing the best and worst performance in advance prevented the biases. Range-presentation is therefore a viable theory-based intervention to improve fairness in serial judgments.