Identifying a species tree subject to random lateral gene transfer

A major problem for inferring species trees from gene trees is that evolutionary processes can sometimes favor gene tree topologies that conflict with an underlying species tree. In the case of incomplete lineage sorting, this phenomenon has recently been well-studied, and some elegant solutions for species tree reconstruction have been proposed. One particularly simple and statistically consistent estimator of the species tree under incomplete lineage sorting is to combine three-taxon analyses, which are phylogenetically robust to incomplete lineage sorting. In this paper, we consider whether such an approach will also work under lateral gene transfer (LGT). By providing an exact analysis of some cases of this model, we show that there is a zone of inconsistency when majority-rule three-taxon gene trees are used to reconstruct species trees under LGT. However, a triplet-based approach will consistently reconstruct a species tree under models of LGT, provided that the expected number of LGT transfers is not too high. Our analysis involves a novel connection between the LGT problem and random walks on cyclic graphs. We have implemented a procedure for reconstructing trees subject to LGT or lineage sorting in settings where taxon coverage may be patchy and illustrate its use on two sample data sets.

► Lateral gene transfer (LGT) can mislead phylogenetic tree inference. ► Exact calculations show a zone of weak statistical inconsistency. ► If LGT rates are low enough then tree inference is still possible. ► Consistency can hold even if each gene is transferred many times on a large tree. ► A tree inference method is described and applied to two data sets.