Communicative modalities for mobile device interaction

•Evaluate communicative modalities when tasking on a mobile device while walking.•A redundant communicative modality was compared to visual and audio only modalities.•We develop an algorithm to quantify error when performing tracing/drawing tasks.•Redundant audio-visual modalities perform comparably to visual modalities.•Modality effects a user's ability to accurately perform tracing tasks.

An investigation of communicative modalities in relation to mobile device interaction while walking is presented. A user evaluation compared three communicative modality conditions: Auditory, Visual, and Mixed (a redundant audio-visual modality). Findings determined that redundant audio-visual modalities are as good as (but no better than) the visual modality, and both are superior to the auditory modality. Reported findings also determined that walking speeds are unaffected by communicative modality.Shape drawing tasks were performed on a touch screen using each modality, and a robust, novel error calculation algorithm was developed to assess the drawing error between the user input and the desired shapes. Drawing error was determined to be significantly higher with the Auditory condition, but drawing speed was unaffected by the communicative modality.The evaluation finds that the visual modality should be leveraged as the primary communicative modality for mobile, map-based interfaces. The drawing error algorithm can be applied to any domain that requires determining precise matchings to known information when drawing.