Within-subject consistency and between-subject variability in Bayesian reasoning strategies

• Two Bayesian reasoning experiments were run.
• Participants adopted consistent, but non-Bayesian, strategies.
• Participants tended to rely on a subset of the information.
• The data for many participants were well accounted for by a linear additive model.

It is well known that people tend to perform poorly when asked to determine a posterior probability on the basis of a base rate, true positive rate, and false positive rate. The present experiments assessed the extent to which individual participants nevertheless adopt consistent strategies in these Bayesian reasoning problems, and investigated the nature of these strategies. In two experiments, one laboratory-based and one internet-based, each participant completed 36 problems with factorially manipulated probabilities. Many participants applied consistent strategies involving use of only one of the three probabilities provided in the problem, or additive combination of two of the probabilities. There was, however, substantial variability across participants in which probabilities were taken into account. In the laboratory experiment, participants’ eye movements were tracked as they read the problems. There was evidence of a relationship between information use and attention to a source of information. Participants’ self-assessments of their performance, however, revealed little confidence that the strategies they applied were actually correct. These results suggest that the hypothesis of base rate neglect actually underestimates people’s difficulty with Bayesian reasoning, but also suggest that participants are aware of their ignorance.