Evolution with stochastic fitnesses: A role for recombination

• We investigate the effect of recombination on the geometric mean principle under different environmental regimens and fitness landscapes.
• We determine invisibility conditions for modifiers of fitness variance.
• We show that only alleles that reduce variance in fitness can invade a population and this invasion threshold is an increasing function of the recombination rate.
• Under some conditions, an allele with a lower geometric mean fitness can achieve highest frequency in the population.

Phenotypic adaptation to fluctuating environments has been an important focus in the population genetic literature. Previous studies have shown that evolution under temporal variation is determined not only by expected fitness in a given generation, but also by the degree of variation in fitness over generations; in an uncertain environment, alleles that increase the geometric mean fitness can invade a randomly mating population at equilibrium. This geometric mean principle governs the evolutionary interplay of genes controlling mean phenotype and genes controlling phenotypic variation, such as genetic regulators of the epigenetic machinery. Thus, it establishes an important role for stochastic epigenetic variation in adaptation to fluctuating environments: by modifying the geometric mean fitness, variance-modifying genes can change the course of evolution and determine the long-term trajectory of the evolving system. The role of phenotypic variance has previously been studied in systems in which the only driving force is natural selection, and there is no recombination between mean- and variance-modifying genes. Here, we develop a population genetic model to investigate the effect of recombination between mean- and variance-modifiers of phenotype on the geometric mean principle under different environmental regimes and fitness landscapes. We show that interactions of recombination with stochastic epigenetic variation and environmental fluctuations can give rise to complex evolutionary dynamics that differ from those in systems with no recombination.