Rethinking testing mode: Should I offer my next chemistry test on paper or computer?

The purpose of this study was to compare student performance on two proctored quizzes and one practice test taken in paper-based or computer-based testing mode using an equal number of algorithmic, conceptual, and definition questions to examine the effect of testing mode when students take multiple tests in class. Data were collected from 221 students enrolled in a general chemistry course at a Midwestern university. After learning the material in lecture, students completed two quizzes: Quiz 1 and Quiz 2. One quiz was delivered on paper and another quiz was taken on computer. Each quiz tested student understanding of different chemistry concepts and the quizzes were proctored on different days. Several days after Quiz 2, students took a practice test that tested the knowledge of Quiz 1 and Quiz 2 with different yet similar questions, and again, the testing mode of the practice test was either paper-and-pencil or computer. After each quiz and the practice test, students received feedback on their performance. Differences in performance between the quizzes and the practice test were measured with normalized gains and the differences between the normalized gains for each condition were analyzed using an Analysis of Variance. The results showed no significant testing mode effects among the four conditions overall, or for algorithmic, conceptual, or definition type of questions. Altogether, the results of the present study do not provide evidence to suggest that instructors need to be concerned about testing mode (paper versus computer) when designing and administering chemistry tests.