Assessing Reflective Writing on a Pediatric Clerkship by Using a Modified Bloom’s Taxonomy

ObjectiveReflection enables learners to analyze their experiences and capture the wisdom that lies within. Effective teaching requires reliable methods of assessment. Several methods of assessing reflective writing have been described; however, they often require significant training, and reliability has seldom been assessed. This study was designed to determine the interrater reliability of a method of assessing reflective writing by using a modified Bloom’s Taxonomy.MethodsTwenty-one third-year medical students maintained reflective journals throughout their pediatric clerkship. A coding schema based on Bloom’s Taxonomy was developed to assess the level of cognitive processing evident in the journals. Journals were independently assessed by 3 raters. Percent agreement, kappa statistics, and intraclass correlation coefficients (ICC [2,1]) were used to assess interrater reliability.ResultsThree hundred eight entries from 21 journals were assessed. Percent agreement ranged from 78.2% to 100%. Kappa statistic for each level ranged from 0.57 ± 0.04 to 0.73 ± 0.04, and for the highest level of processing evident it ranged from 0.52 ± .04 to 0.58 ± 0.04. ICC (2,1) for each level of cognitive processing ranged from 0.62 (F = 6.20; P = .000) to 1.00, and for the highest level of cognitive processing evident, it was 0.79 (F = 12.42; P = .000). Substantial to almost perfect agreement was attained.ConclusionsReflective journals allow learners to revisit their experiences for critical analysis and deeper learning. This study describes a reliable method, based on Bloom’s Taxonomy, of determining whether learners have achieved higher order thinking through reflective journal writing. This method can provide a baseline for facilitating higher order processing, critical thinking, and reflective practice.