A practical illustration of multidimensional diagnostic skills profiling: Comparing results from confirmatory factor analysis and diagnostic classification models

In recent years there has been an increasing international interest in fine-grained diagnostic inferences on multiple skills for formative purposes. A successful provision of such inferences that support meaningful instructional decision-making requires (a) careful diagnostic assessment design coupled with (b) empirical support for the structure of the assessment grounded in multidimensional scaling models. This paper investigates the degree to which multidimensional skills profiles of children can be reliably estimated with confirmatory factor analysis models, which result in continuous skill profiles, and diagnostic classification models, which result in discrete skill profiles. The data come from a newly developed diagnostic assessment of arithmetic skills in elementary school that was specifically designed to tap multiple skills at different levels of definitional grain size.