The determinacy of reproducibility assessments of qualitative microbial food borne pathogen methods detecting a few microbes per analytical portion

A summary of an examination of studies of interlaboratory reproducibility of measurements for detecting the presence of microorganisms in food products is presented. In such studies multiple laboratories, 10 or more, compare the performances of reference and test methods at the limit of detection of the methods that is at spiking levels around 1 cfu per analytical portion. A laboratory's performance is expressed as the number of positive replicates detected per set of six. The data only imply the presence or absence of significant between-laboratory effects with the test method relative to the reference method. It is difficult to parse the observed variability into the contributions of the sample variability and between-laboratory effects. This is because at spiking levels close to 1 cfu per portion it cannot be assured that laboratories are examining portions with equivalent numbers of the target microbe. In this study published data are reformulated to the number of laboratories observing a given number of positive results per replicate set in order to reflect the inhomogeneity of the spike distribution in the replicate portions. A mean spiking level that is less uncertain than the reported 3-tube reference method MPN value is estimated from the pooled proportions of positives that the laboratories obtained with the reference method. The expected distribution of the spike was calculated from its mean value using the binomial equation. The numerical distributions of the laboratories among the 7 possible positive categories (0-6 positives per replicate set) were statistically indistinguishable from the expected binomial distribution thus corroborating this approach. Probable instances of interlaboratory performance differences were detected by further statistical analysis. This analytical approach, as well as transparently reflecting the sampling variability, also suggested ways of improving and simplifying such studies.

► A different way of treating qualitative microbiology method validation is presented. ► It takes into account the high sample variability at low microbe levels. ► It allows detection of laboratory irreproducibility. ► It indicates possible improvements to the method validation process.