Reinforcement learning with Marr

- Reinforcement learning as a field spans all three of Marr's levels of analysis.
- Despite much progress, open questions remain at every level.
- These call for multidisciplinary research that crosses boundaries between levels.

To many, the poster child for David Marr's famous three levels of scientific inquiry is reinforcement learning - a computational theory of reward optimization, which readily prescribes algorithmic solutions that evidence striking resemblance to signals found in the brain, suggesting a straightforward neural implementation. Here we review questions that remain open at each level of analysis, concluding that the path forward to their resolution calls for inspiration across levels, rather than a focus on mutual constraints.