Head movement and facial expressions as game input

• We present a game that utilizes information on face position and facial expressions.
• A user study was conducted to evaluate the game.
• The utilization of face analysis enhanced players’ game experiences in many ways.
• Facial information detected by computer vision offers a promising way for game control.

This study aimed to develop and test a hands-free video game that utilizes information on the player’s real-time face position and facial expressions as intrinsic elements of a gameplay. Special focus was given to investigating the user’s subjective experiences in utilizing computer vision input in the game interaction. The player’s goal was to steer a drunken character home as quickly as possible by moving their head. Additionally, the player could influence the behavior of game characters by using the facial expressions of frowning and smiling. The participants played the game with computer vision and a conventional joystick and rated the functionality of the control methods and their emotional and game experiences. The results showed that although the functionality of the joystick steering was rated higher than that of the computer vision method, the use of head movements and facial expressions enhanced the experiences of game playing in many ways. The participants rated playing with the computer vision technique as more entertaining, interesting, challenging, immersive, and arousing than doing so with a joystick. The results suggested that a high level of experienced arousal in the case of computer vision-based interaction may be a key factor for better experiences of game playing.