Identifying students’ mathematical skills from a multiple-choice diagnostic test using an iterative technique to minimise false positives

There is anecdotal evidence that a significant number of students studying computing related courses at degree level have difficulty with sub-GCE mathematics. Testing of students’ skills is often performed using diagnostic tests and a number of computer-based diagnostic tests exist, which work, essentially, by testing one specific diagnostic skill at a time.This paper proposes using a multiple-choice computer-based diagnostic test where each question has a number of diagnostic skills associated with it in order to allow more flexible questions. A simple measure of a diagnostic skill’s competency could be obtained by calculating the number of questions answered correctly, divided by the total number of questions, associated with that skill. However, because a question may have many skills associated with it, if a question is wrong, then each skill is deemed to be problematic, even though there may not be a problem with all skills.A technique has been developed that refines the initial skill competencies and iteratively re-calculates the skills based on all other competencies. Pilots of the new diagnostic test with first year computing students indicate that particular mathematical problems exist for many students, and suggest that the iterative algorithm produces a more precise indication of competencies than a simple competence measure approach.