https://d2ng68cf36kxsk.cloudfront.net/store/public/post/898/post_images/Slide308.JPG">

These finely modulated sound differences would be detectable on a spectrograph once such a thing had been invented.²²⁷ As early as 1923 Jakobson proposed a new science, “phonology,” which he defined as “the study of speech sounds with regard to meaning and … the strictly relational character of the sense-discriminative entities, linked to each other by binary oppositions as components of the ever-hierarchical phonological systems.”²²⁸

Then, “[w]hile phonetics seeks to collect the most exhaustive information on gross sound matter, in its physiological and physical properties, phonemics, and phonology in general, intervenes to apply strictly linguistic criteria to the sorting and classification of the material registered by phonetics.”²²⁹ These, he said, were the “fundamentals of language” – a science of system and order in phonemes, syllables, and prosodic features such as tone and force, pitch, loudness, and duration.

To the structuralist point, these features made no sense in isolation: “Saussure rightly emphasised that the important thing as far as phonemes are concerned is not at all each phoneme’s individual phonic quality considered in isolation and existing in its own right. What matters is their reciprocal oppositions within a phonological system.”²³⁰ Following Saussure, the sound signifiers were arbitrary in relation to the things they signified.²³¹ After this insight, in 1928, a new organizing concept for the theoretical repertoire of the Prague Linguistics Circle: “structural linguistics.”²³²

For Jakobson, the atomic units of language are phonemes, qualified by however many other voiceable overlays. Today, these phonemes are codified in the International Phonetic Alphabet.²³³ Supported in Unicode, this is unequivocally text, albeit an attempt at comprehensive transliteration of the speech of any language. In the International Phonetic Alphabet, the Roman alphabet is extended to 107 characters, modified by 31 diacritics, and 19 characters to represent tone, stress, and intonation. The phonetic alphabet is only for linguistic aficionados and even then, it is nowhere near enough to capture the full gamut of variation in speech.

Maintained by the International Phonetic Association (or in its own script, the ɪntə‘næʃənəl fə‘nɛtɪk əsoʊsi’eɪʃn), the alphabet was founded in 1886 and is today managed by a private company directed by academics and registered in Britain. Between Unicode and the International Phonetic Alphabet, here is yet another instance of the strangely haphazard governance of our means of production of meanings in the era of digitally-mediated representation and communication.

Not merely a distinct science, Jakobson says, phonemics is the basis of all language. But how so, if text is so different, as we have been arguing? (But half the world is illiterate, Jakobson says.) How so for Chinese? (But its readers still hear the words if they say these to themselves as they read, he says.) How so the sign languages created in deaf communities, which are primarily ideographic?^§AS1.4.3 (But this is only one person in a thousand, he says.)²³⁴ And what about reading itself, where phonemic sounding, aloud or silently, is too slow for meaningful comprehension? Instead we read in “saccades” where our eye jumps then rests for an instant on syllables or words, and in this way alphabetical text effectively becomes ideographic. None of these counter arguments seems strong enough for Jakobson to undo his conflation of speech and text into language, and then its reduction to phonemics. But they are strong enough for us.

When does phonemics reach its limits? When are the differences in affordance so great that transliteration is impossible and multimodality becomes essential? Here we will rely on an example of Jakobson’s. “A former actor of Stanislavskij’s Moscow Theater told me how at his audition he was asked by the famous director to make forty different messages from the phrase Segodnja vecerom, ‘this evening,’ by diversifying its expressive tint. He made a list of some forty emotional situations, then emitted the given phrase in accordance with each of these situations, which his audience had to recognize only from the changes in the sound shape of the same two words.”²³⁵ No text could capture this, not even the International Phonetic Alphabet.

Then there is Jakobson’s other great love, poetry, as parsed by structural linguistics: “Versification, with its diaphanous dichotomies of downbeat-upbeat, break-bridge, and with its correlation of two fundamental metrical concepts, namely design and instance, offered the self-evident possibility of determining the relational invariance that the verse retains across its fluctuations, and of defining and interpreting the scale of the latter.”²³⁶

However, we would retort, for its premeditation and unavoidably multilinear design processes, poetry is in the first instance and quintessentially a textual product, though anticipating, of course, its sound shape at the point of recitation or reading. Speech finds its canonical expression in spontaneous, unscripted, self-to-other, or self-to-self utterance. Poetry is a hybrid, a creature of multimodal transposition, a multimedia materialization.

And now to the era of digitized sounds. A first generation of computer scientists and computational linguists attempted to implement voice synthesis with computer on the basis of transposition of text to phonemic units (characters, diphthongs, syllables).²³⁷ This has in most instances today been supplanted by breaking up text into n-grams, words and phrases recorded by voice actors – in other words, contiguous phonemes or syllables that are distinguishable in text as ideographs.

Speech Synthesis Markup Language is a World Wide Web Consortium’s standard that ties these processes together across different platforms.²³⁸ This change of approach directly parallels what was in the area of text, the shift from structuralist approaches to machine translation in the Chomskyan mold, to statistical machine translation.^§AS2.2.1a

So when GPS^§AS1.3.2b today speaks us on our way, the source of its meanings and the units of its rendering are semantically defined synthetic units that have been recorded as whole spoken words, and aligned in another multimodal transposition to the geocoordinates of space. The sounds of synthetic speech are not kept as a small number of phonemes, but in huge databases with very large numbers of whole words and phrases, ready for transposition of digitized representations of space into text or speech. The unit of storage is the sounded equivalent of the ideograph, not the phoneme.

Phonemic structures may offer an account for the microdesign of speech in the terms Jakobson described in his structural linguistics, but functionally speaking there is no meaning in these. Or to return to Saussure’s terminology, at this level we have things that look like signifiers but without a signified. Today, phonemics have become less relevant or helpful than ever for managing the transpositions of text to speech or other meanings. Or, as Jack Goody has told us,^§MS0.2.1a we’re all turning Chinese.

---

²²¹ Roman Jakobson, 1992 [1997], My Futurist Years, translated by S. Rudy, New York: Marsilio Publishers, p. 39.

²²² Ibid., pp. xvi, 154.

²²³ F.W. Galan, 1985, Historic Structures: The Prague School Project, 1928–1946, Austin: University of Texas Press.

²²⁴ Roman Jakobson, 1992 [1997], My Futurist Years, translated by S. Rudy, New York: Marsilio Publishers, p. x.

²²⁵ Roman Jakobson. 1976 [1978], Six Lectures on Sound and Meaning, translated by J. Mepham, Cambridge, MA: MIT Press, pp. xi–xii.

²²⁶ Roman Jakobson and Linda R. Waugh, 1979 [2002], The Sound Shape of Language, Berlin: Mouton de Gruyter.

²²⁷ Roman Jakobson, 1972, “Verbal Communication,” Scientific American 227(3):72–81, p. 74.

²²⁸ Roman Jakobson, 1980 [1985], Verbal Art, Verbal Sign, Verbal Time, Minneapolis: University of Minnesota Press, p. 4.

²²⁹ Roman Jakobson and Morris Halle, 1956, Fundamentals of Language, The Hague: Mouton, p. 7.

²³⁰ Roman Jakobson, 1976 [1978], Six Lectures on Sound and Meaning, translated by J. Mepham, Cambridge, MA: MIT Press, p. 76.

²³¹ Roman Jakobson, 1971, Selected Writings, Volume 2: Word and Language, The Hague: Mouton, p. 345.

²³² Roman Jakobson, 1970, Main Trends in the Science of Language, New York: Harper and Row, p. 13.

²³³ International Phonetic Association, 1999, Handbook of the International Phonetic Association, Cambridge: Cambridge University Press.

²³⁴ Roman Jakobson and Linda R. Waugh, 1979 [2002], The Sound Shape of Language, Berlin: Mouton de Gruyter, p. 74.

²³⁵ Roman Jakobson, 1960, “Closing Statement: Linguistics and Poetics,” pp. 350–77 in Style in Language, edited by T.A. Sebeok, Cambridge, MA: MIT Press, p. 354.

²³⁶ Roman Jakobson, 1980 [1985], Verbal Art, Verbal Sign, Verbal Time, Minneapolis: University of Minnesota Press, p. 3; Richard Bradford, 1994, Roman Jakobson: Life, Language, Art, London: Routledge.

²³⁷ Jonathan Allen, M. Sharon Hunnicutt, and Dennis Klatt, 1987, From Text to Speech: The Mitalk System, Cambridge: Cambridge University Press.

²³⁸ www.xml.com/pub/a/2004/10/20/ssml.html

---

Kalantzis, Mary, and Bill Cope. 2020. Adding Sense: Context and Interest in a Grammar of Multimodal Meaning. Cambridge UK: Cambridge University Press. pp. 95–100. https://doi.org/10.1017/9781108862059

Roman Jakobson’s “Sound Shape of Language” §AS1.4.1a