Effects of selection and mutation on mitochondrial variation and inferences of historical population expansion in a Caribbean reef fish

Mitochondrial sequence data has contributed to the understanding of historical demography through the application of neutrality tests and coalescent estimators of population growth rates. Characteristics of the mitochondrial genome, such as high mutation rate and lack of recombination, render it particularly suitable for these types of studies. However, selection can also affect patterns of mitochondrial variation. Furthermore, conclusions based on single mitochondrial loci can be sensitive to differences among mitochondrial genes in mutation rate and pattern and levels of homoplasy. We tested the contributions of these factors to patterns of mitochondrial variation in the Caribbean reef fish Halichoeres bivittatus using a multilocus sequence-based approach. Mitochondrial protein-coding loci deviated strongly from a neutral model of evolution and indicated high rates of estimated population growth. In contrast, the mitochondrial control region and a nuclear intron showed little or no deviation from neutrality and low estimated growth rates. The level of variation among loci is inconsistent with a demographic explanation and likely stems from the influence of mutation and selection on the mitochondrial genome. In H. bivittatus, a finding of high rates of population growth is likely an artifact of selection on mitochondrial proteins. This result suggests caution in the interpretation of variation at single mitochondrial loci, and highlights the importance of the use of unlinked nuclear loci to test demographic inferences made from mitochondrial DNA.

Research highlights► Mitochondrial protein-coding genes in Halichoeres bivittatus deviate from neutrality and indicate high estimated population growth rates. ► The mitochondrial control region and a nuclear intron show little deviation from neutrality and significantly lower growth rates. ► Mitochondrial protein-coding genes are under strong purifying selection and may show traces of positive selection. ► Homoplasy in the control region is due to the presence of hypermutable sites.