Assessment of physician competency in patient education: Reliability and validity of a model-based instrument

ObjectiveEstablish the inter-rater reliability and the concept, convergent and construct validity of an instrument for assessing the competency of physicians in patient education.MethodsThree raters assessed the quality of patient education in 30 outpatient consultations with the CELI instrument. This instrument is based on a goal-directed model of patient education and assesses distinctive skills for patient education categorized in four subcompetencies. The inter-rater reliability was calculated. The concept validity was explored by factor analysis. The convergent validity was established by a comparison with two measures of patient-centred behaviour. The construct validity was explored by relating the subcompetencies with physician gender and patient satisfaction.ResultsThe inter-rater reliability for the subcompetencies varied between 0.65 and 0.91. The factor analysis distinguished the four subcompetencies. All subcompetencies correlated with the measures of patient-centred behaviour. Female physicians performed better than male physicians on three subcompetencies. Positive correlations were found for three subcompetencies and patient satisfaction.ConclusionThe CELI instrument appears to be a reliable and valid instrument. However, further research is needed to establish the generalizability and construct validity.Practice implicationThe CELI instrument is a useful tool for assessment and feedback in medical education since it assesses the performance of distinctive skills.