The Assessment of Physiotherapy Practice (APP) is a reliable measure of professional competence of physiotherapy students: a reliability study

QuestionWhat is the inter-rater reliability of the Assessment of Physiotherapy Practice (APP) instrument, and what is the error associated with individual scores?DesignCross-sectional inter-rater reliability study. Thirty pairs of clinical educators each assessed one student after observing student practice over a 5-week clinical placement.ParticipantsSixty clinical educators from five Australian universities formed 30 independent pairs of assessors.Outcome measuresEach pair completed two independent assessments of one student, providing 60 completed APP assessments and an associated Global Rating Scale score for analysis.AnalysisCorrelational coefficients and measurement error expressed in APP scale units were computed to provide a comprehensive analysis of the likely utility of APP scores and to enable score and change score interpretation.ResultsPercentage of agreement between assessors for each item ranged from 56% (Item 19, evidence-based practice) to 83% (Item 20, risk management) and across all items averaged 70% (SD 7). The ICC(2,1) was 0.92 (95% CI 0.84 to 0.96) for the total APP score and 0.72 (95% CI 0.50 to 0.86) for the Global Rating Scale. The standard error of measurement for the total score (scale width 0-80) was 3.2 APP points and the MDC90 was 7.86 representing 9% of the scale width. Bland-Altman analyses identified no systematic differences between raters.ConclusionClinical educators demonstrated a high level of reliability when using the APP instrument to assess physiotherapy students' level of professional competence in workplacebased practice. [Dalton M, Davidson M, Keating JL (2012) The Assessment of Physiotherapy Practice (APP) is a reliable measure of professional competence of physiotherapy students: a reliability study.Journal of Physiotherapy58: 49-56]