The effects of using answer sheets on reported drug use and data quality in a classroom survey: A cluster-randomized study

- Response mode in self-administered surveys can significantly affect data quality.
- Using a separate answer sheets decrease data quality among adolescents.
- Low data quality can distort substance use estimates and trends.

BackgroundWe compare self-reported prevalence of drug use and indicators of data quality from two different response modes (with and without an independent answer sheet for recording responses) in a survey conducted in 2015 among secondary school students.MethodsStratified cluster-randomized study conducted among students in grades 8-12 from public, private and subsidized schools in Chile (N = 2317 students in 122 classes). Measurements included were: percentage reporting substance use (tobacco, alcohol, marijuana, cocaine, ecstasy); number of inconsistent responses; number of item nonresponses; percentage of extreme reports of drug use; percentage reporting using the nonexistent drug, relevón; and completion times.ResultsCompared with those who responded directly in the questionnaire booklet, students who used a separate answer sheet took 17.6 more minutes (95% confidence interval [CI]: 14.4-20.8) to complete the survey and had on average 1.5 more inconsistent responses (95%CI: 0.91-2.14). The prevalence and variance of drug use was higher among those who used an answer sheet for all substances except tobacco; the prevalence ratio (PR) of reported substance use for low-prevalence substances during the past year were: cocaine PR = 2.5 (95%CI: 1.6-4.1); ecstasy PR = 5.0 (95%CI: 2.4-10.5); relevón PR = 4.8 (95%CI: 2.5-9.3).ConclusionsUsing an answer sheet for a self-administered paper-and-pencil survey of drug use among students result in lower quality data and higher reports of drug use. International comparison of adolescent drug use from school-based surveys should be done with caution. The relative ranking of a country could be misleading if different mode of recording answers are used.