https://d2ng68cf36kxsk.cloudfront.net/store/public/post/1196/post_images/Fig.%201.5%28i%29.png">

Fig. 1.5(i): Endophoric relations (structure) and exophoric relations (context)

The logic of endophoric relations and exophoric associations, we find, often look and sound the same. “This” can refer to something earlier in a spoken utterance (structure: cohesion), or something outside of it (context). Grammatically, however, the two “this”s are profoundly different.

We classified structural relations loosely as kind-of, part-of, and causes.^§MS3.3 Halliday and Hasan call these relations synonymy (repetition); antonymy (paired opposites); hyponymy (kind-of); meronomy (part-of); conjunction (an ambiguous range including cause); and ellipsis (meaningful absence). They can look back across time with its sequential unfolding of media and the experiences these media depict. This they do by way of retrospective association (anaphora), or forward by way of anticipatory association (cataphora).⁴⁸⁰ These grammatical concepts can be applied, not only to text and speech, but to image, space, object, body, and sound as well.

The same devices that work for endophoric structure within the frame also work to make meaningful exophoric connections beyond the frame.⁴⁸¹ Of course, pushing them out to context profoundly changes the nature of their work of meaning, and this is why for the purposes of this grammar we make the separation between relations in structure and associations in context. If our frame is a book, words, sentences, paragraphs, and chapters sit in part-of relations to each other, but when a book is part-of a library or a bookstore, these are quite different kinds of association. Vectors many establish causal relations in an image, but the potential causal associations in the display of an image are quite different.

Now, to certain kinds of association in context. For a multimodal grammar, we identify three: seriality, scaling, and expressive causality. Seriality is a function of meanings that are associated by juxtaposition, often because they are a kind-of. Scaling is a function of meanings that fit inside each other, because the one is part-of the other. If we can track immediate causality in structure, expressive causality is a function of the wider context, in its multiplicity, where a framed meaning is symptomatic of a social structure, a culture, a way of life, or an historical era.

Fig. 1.5(ii) shows some roughly categorized examples of seriality, scaling, and expressive causality across the canonical meaning forms.

Fig. 1.5(ii): Some examples of seriality, scaling, and causality

Jean-Paul Sartre^§MS3.1.2b contrasts three kinds of social collocation, illustrating for us this seriality/scaling/expressive causality distinction. A group of people, he says, are waiting for a bus in front of the church in Place Saint-Germain, Paris. They are together in this moment as a consequence of “the inert effect of separate activities.” This, we would term a serial association, the association of bodies in a crowd. “[W]e are concerned here with a plurality of isolations: these people do not care about or speak to each other and, in general, do not look at one another.” Such is “the practico-inert ensemble within which there is a movement towards interchangeability … of the instrumental ensemble.” This does not mean lack of association, which Sartre calls “serial unity,” even if it is a “provisional negation of their reciprocal obligation to Others.”

Contrast this with these people’s actual social relations, Sartre suggests. These may well be on their mind at the time, waiting for the bus – their families, their bosses, in other words “social structures in so far as they express the fundamental social order.” Such meanings involve scaling, from the embodied person to social structures; and expressive causality where instances of meaning are expressive of what Sartre calls a “totality.”

Sartre goes on to speak of collectives as the basis for a mapping of social causes, the social meanings made by soldiers in armies, workers in unions, anti-Semites in racist movements – each is an “organized form of social relations.” The collective then associates by expressive causality, as a consequence of which “plurality becomes unity, alterity becomes my own spontaneity, … a human relation of reciprocity.”⁴⁸² Perhaps even, the serially associated bodies at the bus stop might be inspired to take political action in support of public transport. In our terms, these are vectors of transposition, the ever-present possibility, immanent as well as imminent, of transposition in contextual association between serial juxtaposition, hierarchical scaling, and collective activation or expressive causality. Critical interpretations of seriality, scaling, and expressive causality may become the bases for alternative design analyses. They may become reasons to (re)design.

And another example of association in context: timescaling.⁴⁸³ I am walking through a gallery that steps through an artist’s development, or a natural history museum that chronicles the evolution of life on earth. I am reading a biography or an historical novel. I experience these things serially. Moments in each timeline are nested within the timeframe. These are parallel scales of time, the one compressed scale (my gallery or museum perambulations, my reading), transposing for another (natural history, life history). These may also be taken as expressive causalities, representing the totalities of natural history, a symptomatic life in the culture, or an historical era. In these ways, associations in context are always and necessarily on the move, between seriality, scaling, and expressive totality.

---

⁴⁷⁹ Gunther Kress, 2014, “What Is Mode?,” pp. 60–75 in The Routledge Handbook of Multimodal Analysis, edited by C. Jewitt, London: Routledge, p. 73; Erving Goffman, 1974, Frame Analysis: An Essay on the Organization of Experience, Cambridge, MA: Harvard University Press.

⁴⁸⁰ M.A.K. Halliday and Ruqaiya Hasan, 1985, Language, Context, and Text: Aspects of Language in a Social-Semiotic Perspective, Geelong, Australia: Deakin University Press, pp. 74–95; M.A.K. Halliday and Ruqaiya Hasan, 1976, Cohesion in English, London: Longman.

⁴⁸¹ Leonard Talmy, 2017, The Targeting System of Language, Cambridge, MA: MIT Press.

⁴⁸² Jean-Paul Sartre, 1960 [1976], Critique of Dialectical Reason, translated by A. Sheridan-Smith, London: Verso, pp. 256–57, 261, 268–69.

⁴⁸³ Ron Scollon and Suzie Wong Scollon, 2014, “Multimodality and Language: A Retrospective and Prospective View,” pp. 205–16 in The Routledge Handbook of Multimodal Analysis, edited by C. Jewitt, London: Routledge; Jay L. Lemke, 2000, “Across the Scales of Time: Artifacts, Activities, and Meanings in Ecosocial Systems,” Mind, Culture and Activity 7(4):273–90.

---

Kalantzis, Mary, and Bill Cope. 2020. Adding Sense: Context and Interest in a Grammar of Multimodal Meaning. Cambridge UK: Cambridge University Press. pp. 177–181. https://doi.org/10.1017/9781108862059

Association §AS1.5