Prediction error in reinforcement learning: A meta-analysis of neuroimaging studies

• A synthesis of prediction error (PE) data in human fMRI reinforcement learning.
• Brain regions coding for PE in instrumental and Pavlovian learning are compared.
• Ventral striatum is more involved in instrumental than Pavlovian learning.
• Separate neural systems compute reward and aversive prediction errors.
• Validity of computational models in fMRI reinforcement learning is reviewed.

Activation likelihood estimation (ALE) meta-analyses were used to examine the neural correlates of prediction error in reinforcement learning. The findings are interpreted in the light of current computational models of learning and action selection. In this context, particular consideration is given to the comparison of activation patterns from studies using instrumental and Pavlovian conditioning, and where reinforcement involved rewarding or punishing feedback. The striatum was the key brain area encoding for prediction error, with activity encompassing dorsal and ventral regions for instrumental and Pavlovian reinforcement alike, a finding which challenges the functional separation of the striatum into a dorsal ‘actor’ and a ventral ‘critic’. Prediction error activity was further observed in diverse areas of predominantly anterior cerebral cortex including medial prefrontal cortex and anterior cingulate cortex. Distinct patterns of prediction error activity were found for studies using rewarding and aversive reinforcers; reward prediction errors were observed primarily in the striatum while aversive prediction errors were found more widely including insula and habenula.