Reinforcement learning and human behavior

- Standard RL explains some aspects of operant learning and its underlying neural activity.
- Nevertheless, some operant learning behaviors seem inconsistent with standard RL.
- Inferring a world model is an important part of state-based learning.
- Direct parametric policy learning bypasses the need to learn the model of the world in terms of what are the relevant states-action pairs.

The dominant computational approach to model operant learning and its underlying neural activity is model-free reinforcement learning (RL). However, there is accumulating behavioral and neuronal-related evidence that human (and animal) operant learning is far more multifaceted. Theoretical advances in RL, such as hierarchical and model-based RL extend the explanatory power of RL to account for some of these findings. Nevertheless, some other aspects of human behavior remain inexplicable even in the simplest tasks. Here we review developments and remaining challenges in relating RL models to human operant learning. In particular, we emphasize that learning a model of the world is an essential step before or in parallel to learning the policy in RL and discuss alternative models that directly learn a policy without an explicit world model in terms of state-action pairs.