A second look at the final number of cells in a fluctuation experiment

- Variation in the final number of cells (Nt) is a concern only when the coefficient of variation (CV) for Nt is large.
- A majority of published experiments exhibit a small CV for Nt.
- The CV for Nt can be reduced by appropriately choosing the inoculum size N0.
- With large N0, existing likelihood-based methods work properly without modification.
- The gamma mixture model is the most robust model to account for excessive variation in Nt.

In a fluctuation experiment, the number of cells existing in a culture immediately before plating (commonly known as Nt) varies across the parallel cultures. However, most existing mathematical models for fluctuation assay data do not recognize the variation in Nt. Despite repeated attempts in the past to integrate this source of variability in the estimation of microbial mutation rates, several questions of practical importance remain unanswered. The present investigation finds that the variation needs accounting for only when the coefficient of variation for Nt is large, and experimental data suggest that the coefficient of variation is often moderate or small. Moreover, an increase in the inoculum size can reduce the coefficient of variation. Through extensive simulation, several existing methods that accommodate the variation in Nt are compared. It was found that a newly devised likelihood method based on the existing gamma mixture model outperforms other existing methods. The investigation focuses on the estimation of mutation rates using the Lea-Coulson model, under which mutation is selectively neutral; however, the paper also explores the major findings' implications for the comparison of mutation rates using the likelihood ratio test, and for the estimation of mutation rates using the Mandelbrot-Koch model that allows for non-neutral mutations.