Invasion fitness for gene-culture co-evolution in family-structured populations and an application to cumulative culture under vertical transmission

Human evolution depends on the co-evolution between genetically determined behaviors and socially transmitted information. Although vertical transmission of cultural information from parent to offspring is common in hominins, its effects on cumulative cultural evolution are not fully understood. Here, we investigate gene-culture co-evolution in a family-structured population by studying the invasion fitness of a mutant allele that influences a deterministic level of cultural information (e.g., amount of knowledge or skill) to which diploid carriers of the mutant are exposed in subsequent generations. We show that the selection gradient on such a mutant, and the concomitant level of cultural information it generates, can be evaluated analytically under the assumption that the cultural dynamic has a single attractor point, thereby making gene-culture co-evolution in family-structured populations with multigenerational effects mathematically tractable. We apply our result to study how genetically determined phenotypes of individual and social learning co-evolve with the level of adaptive information they generate under vertical transmission. We find that vertical transmission increases adaptive information due to kin selection effects, but when information is transmitted as efficiently between family members as between unrelated individuals, this increase is moderate in diploids. By contrast, we show that the way resource allocation into learning trades off with allocation into reproduction (the “learning-reproduction trade-off”) significantly influences levels of adaptive information. We also show that vertical transmission prevents evolutionary branching and may therefore play a qualitative role in gene-culture co-evolutionary dynamics. More generally, our analysis of selection suggests that vertical transmission can significantly increase levels of adaptive information under the biologically plausible condition that information transmission between relatives is more efficient than between unrelated individuals.