A modelling approach to predict the variation of repeatability and reproducibility of a RT-PCR assay for infectious salmon anaemia virus across infection prevalences and infection stages

Conventional studies on the precision of diagnostic tests with binary outcomes report single descriptive estimates of agreement for a particular pool of samples. However, agreement for binary tests is intrinsically associated with the assay operating characteristics that are influenced by population and laboratory covariate factors. Therefore, reporting agreement estimates under various conditions may be more appropriate for diagnostic test comprehension. In this study, the influence of various submission factors (tissue sample homogenization, prevalence of infection and pathogen level) on agreement was further analyzed using test result information from a previous descriptive report of within and between laboratories agreement (repeatability and reproducibility, respectively) of a Reverse-Transcription Polymerase Chain Reaction (RT-PCR) assay for infectious salmon anaemia virus (ISAV). Multilevel logistic regression models were constructed separately for non-, low- or high-infected salmon (classified using a study pseudogold standard) to predict probabilities of testing positive under different testing conditions. For each of the 3 infection categories, agreement and kappa values within and between laboratories were computed from the models' predicted values using probability formulae. Thereafter, overall estimates were predicted using simple category weighting for various proportions of infection stages. Agreement varied substantially among infection categories and, consequently, overall repeatability and reproducibility varied greatly with prevalence. This confirmed that the report of a single descriptive estimate (corresponding to a set prevalence) may not be appropriate. Low-infected fish had the lowest agreement estimate which was improved by sample homogenization. This supported a heterogeneous distribution of ISAV in early infected salmon kidney. However, tissue homogenization increased the probability to obtain a false-positive test result (cross-contamination suspected) and decreased agreement in non-infected fish. Compared to conventional report of test agreement estimation, the modelling approach identified influencing submission factors and provided predictive intervals of agreement that give a better expectation and understanding of assay repeatability and reproducibility under different circumstances of use.