The application of temporal difference learning in optimal diet models

Author-Highlights
• We apply model-free reinforcement learning to optimal diet models.
• The presented model incorporates uncertainty of changing environments.
• The model predicts effects of Batesian mimics and aposematism on predators diet choice and energy intake.
• The model uses a precondition of exploration of the action space for successful aversion formation.
• Conflicting rewards lead to foraging behaviour which is conditionally suboptimal in fixed environments but allows better adaptation in changing environments.

An experience-based aversive learning model of foraging behaviour in uncertain environments is presented. We use Q-learning as a model-free implementation of Temporal difference learning motivated by growing evidence for neural correlates in natural reinforcement settings. The predator has the choice of including an aposematic prey in its diet or to forage on alternative food sources. We show how the predator's foraging behaviour and energy intake depend on toxicity of the defended prey and the presence of Batesian mimics. We introduce the precondition of exploration of the action space for successful aversion formation and show how it predicts foraging behaviour in the presence of conflicting rewards which is conditionally suboptimal in a fixed environment but allows better adaptation in changing environments.