Development of a hybrid mental spelling system combining SSVEP-based brain-computer interface and webcam-based eye tracking

The goal of this study was to develop a hybrid mental speller that can effectively prevent unexpected typing errors in the steady-state visual evoked potential (SSVEP)-based mental speller by simultaneously using the information of eye-gaze direction detected by a low-cost webcam without calibration. In the implemented hybrid mental speller, a character corresponding to the strongest SSVEP response was typed only when the position of the selected character coincided with the horizontal eye-gaze direction ('left', 'no direction', or 'right') detected by the webcam-based eye tracker. When the character detected by the SSVEP-based mental speller was located in the direction opposite the eye-gaze direction, the character was not typed at all (a beep sound was generated instead), and thus the users of the speller did not need to correct the mistyped character using a 'BACKSPACE' key. To verify the feasibility and usefulness of the developed hybrid mental spelling system, we conducted online experiments with ten healthy participants, each of whom was asked to type 15 English words consisting of a total of 68 characters. As a result, 16.6 typing errors could be prevented on average, demonstrating that the proposed hybrid strategy could effectively enhance the performance of the SSVEP-based mental spelling system.