A multimodal discourse theory of visual narrative

• This article sets out a linguistically-based pragmatic model of multimodal discourse.
• The model extends formal dynamic discourse semantics to cover multimodal artefacts.
• A general methodology for analysing multimodal artefacts is proposed on the basis of the model.
• Several examples drawing on comics and graphic novels are analysed in detail.
• Consequences and recommendations for empirical validation are discussed.

There have been many attempts to provide accounts of visually expressed narratives by drawing on our understandings of linguistic discourse. Such approaches have however generally proceeded piecemeal — particular phenomena appearing similar to phenomena in verbal discourse are selected for discussion with insufficient consideration of just what it means to treat visual communication as discourse at all. This has limited discussions in several ways. Most importantly, analysis is deprived of effective methodologies for approaching visual artefacts so that it remains unclear what units of analysis should be selected and how they can be combined. In this paper, we articulate a model of discourse pragmatics that is sufficiently general to apply to the specifics of visually communicated information and show this at work with respect to several central aspects of visual narrative. We suggest that the framework provides an effective and general foundation for reengaging with visual communicative artefacts in a manner compatible with methods developed for verbal linguistic artefacts.