The value of using test response data for content validity: An application of the bifactor-MIRT to a nursing knowledge test

SummaryAimThis paper aimed 1) to argue for the value of using test response data for content validation, and b) to demonstrate this practice using bifactor-multidimensional item response theory (bifactor-MIRT) for nurse education.MethodThe Nursing Knowledge Test (NKT) response data by 1491 nurse students from China were used for demonstration. Based on the content structure assumed by subject-matter experts (SME), a bifactor-MIRT model was constructed and tested. This involved five steps: dimensionality assessment, local dependence detection, model specification, calibrating and unit weighting.ResultsDimensionality assessment results confirmed the content structure assumed by SME. Through local dependence detection and calibrating (i.e., item parameter check), items suspected of contaminating content were detected and those producing substantive harm were removed or constrained. Finally, content contributions by items to the overall scale and to their subscales were obtained through unit weighting.ConclusionDeficiencies residing in SME for content validation must raise attention. The study suggests the value of modeling test response data to compensate these deficiencies. The theoretical implication is discussed.