Usability study of text-based CAPTCHAs

Completely Automatic Public Turing test to tell Computers and Humans Apart, or CAPTCHA, is a security measure that guards a system from exploitation by the discrimination between a real human being and an automated computer program via the method of presenting to the unknown user the challenges that are hard for computer yet easy for human. Focusing on text-based CAPTCHA, this study conducted an experiment to study the effect of age groups and distortion types on the CAPTCHA task. Twenty-four participants were recruited to take part in the experiment, where twelve of them were in the elder group (aged 50–56) and twelve in the young group (aged 23–24). One type with no distortion and five types of common CAPTCHA distortion techniques were considered. The results of non-parametric analysis of the data revealed that the participants of two age groups differed significantly in terms of response time, error rate, and NASA-TLX score. Distortion type had significant effect on response time, error rate, Critical flicker fusion (CFF), and NASA-TLX score. Post-hoc analysis showed that Blot Mask and Line Mask were the hardest CAPTCHAs, while Thread Noise, Global Warp, and Geometry Noise were on a par with Normal Type (no distortion). Dependent variables were also correlated to each other. With the inevitability of the security measure and the increasing population of elder internet users, this study has important implications for the design of CAPTCHA systems.

Research highlights
► Elder and young participants differed in terms of CAPTCHA task performance.
► CAPTCHAs with incoherent and invisible occluding noises were hard to solve.
► Shape ambiguity and phonetic ambiguity may affect the performance of CAPTCHA tasks.