Evaluating antimicrobial therapy: How reliable are remote assessors?

• Many hospitals lack the capacity to assess antimicrobial appropriateness by local assessors with infectious diseases training.
• It is unclear whether assessments of antimicrobial prescriptions are consistent among remote assessors.
• Findings can be used to determine the types of remote assessors that can reliably assess antimicrobial therapy.

IntroductionAssessing the quality of antimicrobial prescribing provides hospitals with a means of targeting and measuring the impact of antimicrobial stewardship interventions. There are limited data available on the reliability of these assessments among different types of hospital assessors deployed away from the bedside (ie remotely). Importantly, it is unclear if assessors inexperienced in clinical infectious diseases can reliably evaluate the quality of antimicrobial prescriptions. This study sought to determine the reliability of assessments made by remote hospital assessors with different levels of clinical infectious diseases experience. These assessments were based on (1) concordance with national prescribing guidelines and (2) ‘overall appropriateness’.Methods180 prescriptions were assessed for ‘concordance with guidelines’ and ‘overall appropriateness’ at the bedside (ie locally). Prescription data were then given to fifteen remote assessors. These assessors were blinded to local assessments. Inter-rater reliability was calculated using Fleiss' kappa statistics.ResultsHigher levels of agreements were achieved for ‘concordance with guidelines’ assessments. Local and remote antimicrobial management teams had the highest level of agreement and this improved when looking at antimicrobial treatment used to treat respiratory tract infections (kappa score = 0.67). Reliability in assessments was moderate for local pharmacist assessments and fair to slight for local infection control assessments.ConclusionsThere is scope to develop tools that will improve assessment reliability of antimicrobial therapy among remote assessors. Clinical pharmacists provide reliability comparable to infectious diseases experts, however, infection control practitioners may require further education and training to improve reliability in assessments.