Best-classifier feedback in diagnostic classification training

•We examined the influence of unequal category payoffs on decision criterion learning with mammography images.•Feedback reflected either objective category membership or the responses of an empirically determined “best” classifier.•Decision criterion values were closer to optimal after best-classifier feedback training.•Results are consistent with the hypothesis that accuracy is overemphasized by learners when payoffs are asymmetric.

Diagnostic classification training requires viewing many examples along with category membership feedback. “Objective” feedback based on category membership suggests that perfect accuracy is attainable when it may not be (e.g., with confusable categories). Previous work shows that feedback based on an “optimal” responder (that sometimes makes classification errors) leads to higher long-run reward, especially in unequal category payoff conditions. In the current study, participants learned to classify normal or cancerous mammography images, earning more points for correct “cancer” than “normal” responses. Feedback was either objective or based on performance of an empirically determined “best” classifier. This approach is necessary because theoretically optimal responses cannot be determined with complex real-world stimuli with unknown perceptual distributions. Replicating earlier work that used simple artificial stimuli, we found that best-classifier performance led to decision-criterion values (β) closer to the reward-maximizing criterion, along with higher point totals and a slight reduction (as predicted) in overall accuracy.