Visualized sound effect icons for improved multimedia accessibility: A pilot study

Sound effects are often used to communicate important information in multimedia such as video games. For instance, they may tell the player that a character has just snuck up on them, is firing at them, or is about to paddle over a waterfall. Nevertheless, there are times when playing sound may be inappropriate, may be inaudible, may become fatiguing and/or may be inaccessible for hard of hearing and deaf users. Therefore, an alternative to sound that can relay the same information would be beneficial to many users. The majority of studies into alternative presentations of sound for these purposes have focused on dialogue at the expense of music and sound effects. The paper introduces a pilot study of “SoundSign”, a prototype symbolic representation of sound effects for multimedia, using an innovative icon and compass that indicates direction, sound cue and proximity. Users who have disabled the sound, are hearing-impaired or are otherwise unable to hear sound will still get the information needed. A description of SoundSign and the results of a usability test are presented.

► Audio accessibility has been neglected but is an important area of research.
► Many types of games need some form of audio accessibility.
► Iconic, symbolic audio accessibility offers a useful alternative to text.
► We developed a system to represent sound effects in games.
► Our tests show that the system was useful when sound was removed from a game.