Augmenting graphical user interfaces with haptic assistance for motion-impaired operators

• Haptic assistance improves computer access for people with motion-impairments.
• Two methods are proposed: deformable haptic cones and deformable virtual switches.
• These were the most effective for reducing missed-clicks and targeting times.
• The results show that they were most effective at reducing the effect of distracters.
• Haptic assistance has been fully automated for any Windows software.

Haptic assistance is an emerging field of research that is designed to improve human–computer interaction (HCI) by reducing error rates and targeting times through the use of force feedback. Haptic feedback has previously been investigated to assist motion-impaired computer users, however, limitations such as target distracters have hampered its integration with graphical user interfaces (GUIs). In this paper two new haptic assistive techniques are presented that utilise the 3DOF capabilities of the Phantom Omni. These are referred to as deformable haptic cones and deformable virtual switches. The assistance is designed specifically to enable motion-impaired operators to use existing GUIs more effectively. Experiment 1 investigates the performance benefits of the new haptic techniques when used in conjunction with the densely populated Windows on-screen keyboard (OSK). Experiment 2 utilises the ISO 9241-9 point-and-click task to investigate the effects of target size and shape. The results of the study prove that the newly proposed techniques improve interaction rates and can be integrated with existing software without many of the drawbacks of traditional haptic assistance. Deformable haptic cones and deformable virtual switches were shown to reduce the mean number of missed-clicks by at least 75% and reduce targeting times by at least 25%.