A model to explain the emergence of reward expectancy neurons using reinforcement learning and neural network

In an experiment of multi-trial task to obtain a reward, reward expectancy neurons, which responded only in the non-reward trials that are necessary to advance toward the reward, have been observed in the anterior cingulate cortex of monkeys. In this paper, to explain the emergence of the reward expectancy neuron in terms of reinforcement learning theory, a model that consists of a recurrent neural-network trained based on reinforcement learning is proposed. The analysis of the hidden layer neurons of the model during the learning suggests that the reward expectancy neurons emerge to realize smooth temporal increase of the state value by complementing the neuron that responds only in the reward trial.