A comparative analysis of multimodal communication during design sketching in co-located and distributed environments

• We characterized multimodal communication during distributed and co-located design sketching.
• Our theoretical framework treats gesturing as a communication channel and a thinking medium.
• When gesturing is restricted, graphical communication might be leveraged to compensate.
• There is a design phase dependent interaction between gestural and graphical communication.
• “Cross-gesturing” relates sketch content of different authors and facilitates shared understanding.

This study extends our understanding of multimodal communication during design sketching. Building on the literature, the theoretical dimension frames gesturing as a communication channel and a thinking medium, and postulates an interplay between gesturing and other channels. The empirical dimension explores the theoretical propositions in the context of co-located and distributed sketching. Quantitative analyses suggest that when gesturing is restricted, graphical communication is leveraged to compensate, and that verbal communication is incessant in both collaboration environments. They also highlight a non-compensatory design phase dependent interaction between gestural and graphical communication. Moreover, they reveal differences in the communication structure used in the two environments. Qualitative analyses identify a behavior termed “cross-gesturing,” which informs how gesturing facilitates shared understanding during collaborative sketching.