Quantitative content analysis procedures to analyse students’ reflective essays: A methodological review of psychometric and edumetric aspects

Reflective essays are a common way to develop higher education students’ reflection ability. Researchers frequently analyse reflective essays based on quantitative content analysis procedures (QCA). However, the quality criteria that should be met in QCA are not straightforward. This article aims to: (1) develop a framework of quality requirements for QCA; and (2) explore the extent to which QCA studies of students’ reflective essays meet these quality requirements. First, a methodological review of psychometric and edumetric criteria for QCA is conducted, resulting in a framework with required quality criteria. Second, 18 studies were selected in which QCA was used to analyse students’ reflective essays. These studies were scrutinised using the developed framework. The results showed that none of the selected studies met all requirements. It is concluded that QCA procedures for analysing students’ reflective essays need improvement and that the developed framework can be used to improve and evaluate these procedures.

► We present a list for developing valid and reliable quantitative content analysis procedures (QCA).
► Eighteen selected articles did not meet all the criteria of the list.
► The list can contribute to improving QCA for evaluating students’ reflective essays.
► Researchers and teachers can introduce QCA as an educational instrument.