Recurrent events counted in evaluations of predictive accuracy

ObjectivesTo describe an alternate approach for the calculation of sensitivity and specificity when analyzing the accuracy of screening tools, which can be used when standard calculations may be inappropriate. SensitivityER (ER denoting event rate) is the number of events correctly predicted, divided by the total number of events. SpecificityER is the amount of time that study participants are predicted to be event negative, divided by the total amount of participant observed time. Variance estimates for these statistics are constructed by bootstrap resampling, taking into account event dependence.MethodsStandard and alternate approaches for calculating sensitivity and specificity were applied to hospital falls risk screening tool data. In this application, the outcome of interest was a recurrent event, there were multiple applications of the screening tool, delays in screening tool completion, and patients' follow-up durations were unequal.ResultsApplication of sensitivityER and specificityER to this data not only provided a clearer description of the screening tool's overall accuracy, but also allowed examination of accuracy over time, accuracy in predicting specific event numbers, and evaluation of the added value that screening tool reapplications may have.ConclusionSensitivityER and specificityER provide a valuable approach to screening tool evaluation in the clinical setting.