Prefrontal executive function and adaptive behavior in complex environments

•PFC function overcomes key adaptive limitations in reinforcement learning (RL).•Model-based RL in PFC provides offline planning abilities adjusting online basic RL.•PFC performs online inferences for gradually building an unlimited set of behaviors.•PFC arbitrates online between adjusting ongoing behavior versus forming new behaviors.•New behaviors stem from learned ones. PFC probes them before consolidating them.

The prefrontal cortex (PFC) subserves higher cognitive abilities such as planning, reasoning and creativity. Here we review recent findings from both empirical and theoretical studies providing new insights about these cognitive abilities and their neural underpinnings in the PFC as overcoming key adaptive limitations in reinforcement learning. We outline a unified theoretical framework describing the PFC function as implementing an algorithmic solution approximating statistically optimal, but computationally intractable, adaptive processes. The resulting PFC functional architecture combines learning, planning, reasoning and creativity processes for balancing exploitation and exploration behaviors and optimizing behavioral adaptations in uncertain, variable and open-ended environments.