The Judgment Bias Task: A flexible method for assessing individual differences in social judgment biases

Many areas of social psychological research investigate how social information may bias judgment. However, most measures of social judgment biases are (1) low in reliability because they use a single response, (2) not indicative of individual differences in bias because they use between-subjects designs, (3) inflexible because they are designed for a particular domain, and (4) ambiguous about magnitude of bias because there is no objectively correct answer. We developed a measure of social judgment bias, the Judgment Bias Task, in which participants judge profiles varying in quality for a certain outcome based on objective criteria. The presence of ostensibly irrelevant social information provides opportunity to assess the extent to which social biases undermine the use of objective criteria in judgment. The JBT facilitates measurement of social judgment biases by (1) using multiple responses, (2) indicating individual differences by using within-subject designs, (3) being adaptable for assessing a variety of judgments, (4) identifying an objective magnitude of bias, and (5) taking 6 min to complete on average. In nine pre-registered studies (N > 9000) we use the JBT to reveal two prominent social judgment biases: favoritism towards more physically attractive people and towards members of one's ingroup. We observe that the JBT can reveal social biases, and that these sometimes occur even when the participant did not intend or believe they showed biased judgment. A flexible, objective, efficient assessment of social judgment biases will accelerate theoretical and empirical progress.