Genetic algorithms and non-ESS solutions to game theory models

Game theory is not only the primary method for the formal modelling of interactions between individuals, but it also underlies how biologists think about social interactions on an intuitive level. Most biologists equate game theory with the notion of an evolutionary stable strategy (ESS). Maynard Smith's definition of an ESS formalized an intuitive concept of stable population equilibria in game theory models. This has proven extremely useful, both to formal modelling efforts, and to a much wider range of biologists in general. The formal definition of an ESS often cannot be met for an entire class of games, extensive form games with nodes off the supported path. Games of this form are the most reasonable models for a great many animal behaviours. This means that formal ESSs will not exist for these games, even though evolutionary stable strategies, in the intuitive sense, do exist. More powerful analytical solution concepts, such as evolutionarily stable sets (ES sets), are required to describe stable endpoints formally, but finding such solutions can be extremely difficult. In this paper we use genetic algorithms as an alternative method of searching for such solutions in a well-studied game of biological communication. Using this technique, we show a previously unknown solution to this game. We discuss the properties of biological communication that make it a particularly difficult subject to model using ESSs as the solution concept, and suggest that these issues also apply to a larger class of social models.