Short CommunicationGene tree rooting methods give distributions that mimic the coalescent process

- In multilocus phylogenetics, gene trees are often estimated separately at each locus.
- For a single true gene tree, multi-locus data were simulated.
- Rooted gene trees were estimated for each locus, and their frequencies recorded.
- The estimated gene tree distribution may be close to the coalescent in this case.
- Species tree estimation based on the coalescent shows variable performance here.

Multi-locus phylogenetic inference is commonly carried out via models that incorporate the coalescent process to model the possibility that incomplete lineage sorting leads to incongruence between gene trees and the species tree. An interesting question that arises in this context is whether data “fit” the coalescent model. Previous work (Rosenfeld et al., 2012) has suggested that rooting of gene trees may account for variation in empirical data that has been previously attributed to the coalescent process. We examine this possibility using simulated data. We show that, in the case of four taxa, the distribution of gene trees observed from rooting estimated gene trees with either the molecular clock or with outgroup rooting can be closely matched by the distribution predicted by the coalescent model with specific choices of species tree branch lengths. We apply commonly-used coalescent-based methods of species tree inference to assess their performance in these situations.