Reinforcement learning, efficient coding, and the statistics of natural tasks

- Reinforcement learning (RL) provides a rich conceptual framework for understanding human learning and decision making.
- Most RL-inspired research in cognitive science and neuroscience has focused on identifying algorithms or procedures.
- We argue for the importance of pursuing the complementary question of how RL problems are represented.
- A foundation for this undertaking is provided by existing work employing the notion of efficient coding.
- In an RL context, efficient coding theory points to exciting new targets for research, including the challenge of understanding the structure of naturalistic tasks.

The application of ideas from computational reinforcement learning has recently enabled dramatic advances in behavioral and neuroscientific research. For the most part, these advances have involved insights concerning the algorithms underlying learning and decision making. In the present article, we call attention to the equally important but relatively neglected question of how problems in learning and decision making are internally represented. To articulate the significance of representation for reinforcement learning we draw on the concept of efficient coding, as developed in perception research. The resulting perspective exposes a range of novel goals for behavioral and neuroscientific research, highlighting in particular the need for research into the statistical structure of naturalistic tasks.