Detecting careless respondents in web-based questionnaires: Which method to use?

• Several methods for detecting careless respondents in survey data were studied.
• Multi-test extensions of person-fit statistics were proposed as an alternative.
• Sensitivity was not optimal for any of the methods.
• Response time and maximum longstrings are recommended as quick checks.
• Person-fit statistics are recommended for detecting inconsistency in response patterns.

High data quality is an important prerequisite for sound empirical research. Meade and Craig (2012) and Huang, Curran, Keeney, Poposki, and DeShon (2012) discussed methods to detect unmotivated or careless respondents in large web-based questionnaires. We first discuss these methods and present multi-test extensions of person-fit statistics as alternatives. Second, we applied these methods to data collected through a web-based questionnaire, in which some respondents received instructions to respond quickly which can result in more careless responding. In addition, we conducted a simulation study. We compared sensitivity and specificity of different methods and concluded that multi-test extensions of person-fit statistics are a good alternative as compared to other methods, although the sensitivity to detect careless respondents using empirical data was lower than using simulated data.