The effect of supplemental instruction on academic performance: An encouragement design experiment

Supplemental Instruction (SI) or PASS (Peer Assisted Study Sessions) has been widely offered to students at tertiary institutions in many countries with the aim of improving academic performance. The SI/PASS evaluation literature is extensive, but it has not adequately addressed potential selection bias. We evaluate an SI/PASS program at an Australian university through a randomized-encouragement-design experiment. A randomly selected subgroup of students from first-year courses (N = 6954) was offered large incentives (worth AUD 55,000) to attend PASS which increased attendance by an estimated 0.47 hours each. This first-stage (inducement) effect did not vary with the size of the incentive and was larger (0.89) for students from disadvantaged backgrounds. Instrumental-variable estimates suggest that 1 hour of PASS improved grades by 0.065 standard deviations, which is consistent with the non-experimental literature. However, this estimate is not statistically significant, reflecting limited statistical power. The estimated effect is largest for students in their first semester at university.