Using Objective Structured Clinical Evaluation for Simulation Evaluation: Checklist Considerations for Interrater Reliability

BackgroundReliability of simulation outcome measurements is infrequently reported in nursing education. The purpose of this study was to establish interrater reliability of a checklist for a pediatric medication administration objective structured clinical evaluation.MethodTwo raters scored 207 videotaped nursing student objective structured clinical evaluation performances using a 14-item checklist. Item interrater reliability statistics were calculated.ResultsAdequate interrater reliability was obtained on six items from the cognitive and psychomotor domains of learning. Unacceptable interrater reliability was obtained on four items from the affective domain.ConclusionResults verified the difficulty of quantitatively measuring affective domain behaviors and the need for consistency in rater roles.