Reinforcement learning and dopamine in the striatum: A modeling perspective

The recent research evidences show that the dopamine (DA) system in the brain is involved in various functions like reward-related learning, exploration, preparation, and execution in goal directed behavior. It is suggested that dopaminergic neurons provide a prediction error akin to the error computed in the temporal difference learning (TDL) models of reinforcement learning (RL). Houk et al. (1995) [26] proposed a biochemical model in the spine head of neurons at the striatum in the basal ganglia which generates and uses neural signals to predict reinforcement. The model explains how the DA neurons are able to predict reinforcement and how the output from these neurons might then be used to reinforce the behaviors that lead to primary reinforcement. They proposed a scheme drawing that parallels between actor–critic architecture and dopamine activity in the basal ganglia. Houk et al. (1995) [26] also proposed a biochemical model of interactions between protein molecules which supports learning earlier predictions of reinforcement in the spine head of medium spiny neurons at the striatum. However, Houk׳s proposed cellular model fails to account for the time delay between the dopaminergic and glutamatergic activity required for reward-related learning and also fails to explain the ‘eligibility trace’ condition needed in delayed tasks of associative conditioning in which a memory trace of the antecedent signal is needed at the time of a succeeding reward. In this article, we review various models of RL with an emphasis on the cellular models of RL. In particular, we emphasize biochemical models of RL, and point out the future directions.