William Cope
Nithin Kalorth
Jako Olivier
Ida Brandão
Mitchell PetersMultimodal Literacies MOOC’s Updates
Section 8.2: Multimodal Meaning and Synaesthesia
Meaning-making is a process of representation (sense-making) and communication (in which a message prompt is interpreted by another person). This update analyses the design process through which people take available resources for meaning, and use these as building blocks for designing new meanings. Although they are often similar, newly made meanings are never quite the same as any previous meanings. These ‘redesigned’ meanings then enter the world. At this point the world has been transformed, even if in the smallest of ways. Then the cycle of meaning-making can start over again. Even though in the schools of our recent past and still today we have separated out literacies as a singular subject that deals with the mechanics of reading and writing, our processes of meaning are always at least to some extent multimodal, bringing together written, visual, spatial, tactile, gestural, audio and oral modes. This is particularly true in the contemporary era. The new, digital media are intrinsically multimodal. This update suggests an approach to ‘design analysis’. It is a prelude to the following sections that outline a multimodal grammar, one that will help us describe and understand meaning-making across different modes.
To explore these issues further, read the supporting material on the Literacies website.
Comment Below: What are the benefits and challenges of a multimodal approach to literacies?
Make an Update: Describe an important site of multimodal communication in your life, or your students lives. How might a multimodal analysis of meaning prove useful? How does this compare with traditional notions of literacy?
My inception to multimodal education
The most prominent site of multimodal communication in my life was introduced to me during my MA days, which was during the first COVID-19 lockdown. Never before was education multi-modal in my life. All my life, education meant reading from the textbook and learning the rest from the teacher in the classroom. As such, reading texts, the images and charts therein, teacher talk in the classroom and the occasional use of blackboard by the teacher to write or simplify complex concepts, were all that defined my education in school. Bachelors wasn't much different, with an occasion video being played by a teacher to enhance something he/she had already explained. The pandemic came, and education across the country underwent a previously unimaginable change: the inclusion of mobile phones, WhatsApp, voice notes, recorded video lectures, Zoom and Teams meetings, Google Classroom, Google Docs, virtual presentations, and what not!
Everyone suddenly needed a stable internet connection, a smartphone or a tablet or laptop to not drop out of the education system. While technology was no alien to me and my peers, the heavy reliance on Google classroom was the game changer. Our notes were uploaded online, the PPTs the teacher taught in the online meetings were accessible on GClass for reference, and we even had to upload our assignments and term papers online. These mostly involved group projects for which we would collaborate online, on a Google Doc. Collaboration and teamwork had an online transition too, and since mine was a teacher-training course, we even had to conduct peer teaching, material designing, and lesson development online! Multimodality was everywhere throughout my MA programme.
Coming to the second question, I think multimodal analysis of meaning is useful because it caters to the multiple intelligences that dominate a classroom. While someone might learn with audio lessons better, another student might find video lessons more interesting, and still another might survive on mindmaps and pictorial descriptions. Having the privilege to choose which mode of learning suits you best is key to making learning useful, fun, and interesting!
To answer the third question, I'll go back to my own academic journey and compare. Learning felt unidimensional in school, and I remember having a lot of average-performing peers back then, but with the inclusion of multimodal literacy, I think a lot of the same peers found their way back to performing better in school, and again, this is linked to the concepts becoming easier and more interesting to them after the shift.
**Benefits of a Multimodal Approach to Literacies:**
1. **Enhanced Engagement:** Multimodal literacies leverage various modes of communication, such as visual, auditory, and kinesthetic elements, making the learning experience more engaging for learners.
2. **Catering to Diverse Learning Styles:** Different students have different learning preferences. A multimodal approach allows educators to address diverse learning styles, ensuring that all students can connect with the material.
3. **Deeper Understanding:** Integrating multiple modes can lead to a more profound understanding of content. For instance, visual aids or hands-on activities can complement textual information, reinforcing key concepts.
4. **Real-World Relevance:** Multimodal literacies reflect the communication modes prevalent in the real world. In today's digital age, being proficient in various modes of communication is essential for effective expression and comprehension.
5. **Inclusivity:** The multimodal approach acknowledges and accommodates diverse abilities and backgrounds. It provides alternative ways for students to demonstrate understanding, fostering a more inclusive learning environment.
**Challenges of a Multimodal Approach to Literacies:**
1. **Resource Intensity:** Developing and implementing multimodal materials can be resource-intensive, requiring access to technology, multimedia resources, and training for educators. This can be a challenge for schools with limited resources.
2. **Assessment Complexity:** Assessing student understanding in a multimodal context may be more challenging than traditional assessments. Educators need to develop appropriate evaluation methods that capture the richness of multimodal projects.
3. **Technological Barriers:** Not all students may have equal access to technology or be familiar with certain digital tools. This can create disparities in students' ability to engage with multimodal learning materials.
4. **Time Constraints:** Integrating multimodal approaches may require additional time in lesson planning and implementation. Teachers might face challenges fitting these approaches into a tightly packed curriculum.
5. **Resistance to Change:** Some educators or institutions may be resistant to adopting new pedagogical approaches. Overcoming resistance and facilitating a smooth transition to multimodal literacy practices can be a challenge.
In conclusion, while a multimodal approach to literacies offers numerous benefits, including enhanced engagement and inclusivity, it comes with challenges related to resource allocation, assessment methods, technological barriers, time constraints, and potential resistance to change. Overcoming these challenges requires thoughtful planning, professional development, and a commitment to creating a dynamic and inclusive learning environment.
I agree with your thoughtful comment on the advantages and challenges attached to multimodal education. While the advantages are game-changing, facing the challenges is the real concern, especially for developing nations where access to technology and tech-enabled teachers is not the same in every region. With a roadmap for development, however, a multimodal approach to communication and education will soon become the new reality.
Based on my experience in the field of elementary school teacher education, especially in the current era, there is a need for multimodal communication. Of course, multimodal can be useful in learning activities. One of the learning programs is the Google Sites site which provides various materials, videos and learning assignments for students. Based on the analysis of multimodal meaning, analysis is needed from students. This can increase student literacy. Students analyze information from writing and videos available on the platform created by the teacher. This can also make students independent in learning both at school and outside school. Students can open the learning site anytime and anywhere. The material provided has also gone through a material analysis process so that the material presented is in accordance with the curriculum. Apart from that, other learning media need to be used to increase students' enthusiasm for learning in class. Students can comment with other students so that active discussion activities occur online. Students can watch videos provided by the teacher either from YouTube or those made by the teacher themselves regarding learning material. There is a feature to save student assignments given by the teacher at each lesson. Students will also receive an evaluation on each material they have studied. With this website, students are given learning challenges so they can move on to the next stage. This is done to increase student learning motivation. Before learning activities begin, students are introduced to language literacy and also numeracy literacy. Teachers are required to be creative in carrying out the learning process from start to finish. The traditional idea of literacy is only related to reading and writing. But currently literacy is related to modality literacy too. This modality can combine several texts, audio, video. Students are able to improve their analysis regarding learning.
Rujukan : Fatimah, C. (2023). Evaluasi Efektivitas Penggunaan Media Pembelajaran Interaktif Dalam Pembelajaran Matematika Di Perguruan Tinggi. Jurnal Dunia Ilmu, 3(1).
I resonate with your comment on the significance of enabling our learners to become motimodal learners whether in terms of language literacy or numeracy literacy. In the 21st century, it is imperative for us, as teachers, to incorporate as many modes of learning as possible so that our learners learn better and become familiar with these different modes, which they will, in all probability, encounter later in their lives!
Based on my experience in the field of elementary school teacher education, especially in the current era, there is a need for multimodal communication. Of course, multimodal can be useful in learning activities. One of the learning programs is the Google Sites site which provides various materials, videos and learning assignments for students. Based on the analysis of multimodal meaning, analysis is needed from students. This can increase student literacy. Students analyze information from writing and videos available on the platform created by the teacher. This can also make students independent in learning both at school and outside school. Students can open the learning site anytime and anywhere. The material provided has also gone through a material analysis process so that the material presented is in accordance with the curriculum. Apart from that, other learning media need to be used to increase students' enthusiasm for learning in class. Students can comment with other students so that active discussion activities occur online. Students can watch videos provided by the teacher either from YouTube or those made by the teacher themselves regarding learning material. There is a feature to save student assignments given by the teacher at each lesson. Students will also receive an evaluation on each material they have studied. With this website, students are given learning challenges so they can move on to the next stage. This is done to increase student learning motivation. Before learning activities begin, students are introduced to language literacy and also numeracy literacy. Teachers are required to be creative in carrying out the learning process from start to finish. The traditional idea of literacy is only related to reading and writing. But currently literacy is related to modality literacy too. This modality can combine several texts, audio, video. Students are able to improve their analysis regarding learning.
Rujukan : Fatimah, C. (2023). Evaluasi Efektivitas Penggunaan Media Pembelajaran Interaktif Dalam Pembelajaran Matematika Di Perguruan Tinggi. Jurnal Dunia Ilmu, 3(1).
your experience underscores the value of incorporating multimodal communication tools like Google Sites in elementary education. This approach not only aligns with the current understanding of literacy but also empowers students to engage with diverse learning materials, fostering independent learning and motivation.
your experience underscores the value of incorporating multimodal communication tools like Google Sites in elementary education. This approach not only aligns with the current understanding of literacy but also empowers students to engage with diverse learning materials, fostering independent learning and motivation.
Based on my experience in the field of elementary school teacher education, especially in the current era, there is a need for multimodal communication. Of course, multimodal can be useful in learning activities. One of the learning programs is the Google Sites site which provides various materials, videos and learning assignments for students. Based on the analysis of multimodal meaning, analysis is needed from students. This can increase student literacy. Students analyze information from writing and videos available on the platform created by the teacher. This can also make students independent in learning both at school and outside school. Students can open the learning site anytime and anywhere. The material provided has also gone through a material analysis process so that the material presented is in accordance with the curriculum. Apart from that, other learning media need to be used to increase students' enthusiasm for learning in class. Students can comment with other students so that active discussion activities occur online. Students can watch videos provided by the teacher either from YouTube or those made by the teacher themselves regarding learning material. There is a feature to save student assignments given by the teacher at each lesson. Students will also receive an evaluation on each material they have studied. With this website, students are given learning challenges so they can move on to the next stage. This is done to increase student learning motivation. Before learning activities begin, students are introduced to language literacy and also numeracy literacy. Teachers are required to be creative in carrying out the learning process from start to finish. The traditional idea of literacy is only related to reading and writing. But currently literacy is related to modality literacy too. This modality can combine several texts, audio, video. Students are able to improve their analysis regarding learning.
Rujukan : Fatimah, C. (2023). Evaluasi Efektivitas Penggunaan Media Pembelajaran Interaktif Dalam Pembelajaran Matematika Di Perguruan Tinggi. Jurnal Dunia Ilmu, 3(1).
An important site of multimodal communication I am interested in now is online news report. As we know that news report can be a good source of multimodality. What interests me is how news agencies present the content (their writing mode) and images. Normally these two modes should be presented in the same direction. In other words, the selected images should support the content. However, it is worth asking a question “Is it possible that the modes used in news report do not support each other?” Recently, there is a movement of a new generation of pro-democracy activists in Thailand and this movement has become controversial in public media. The activists mainly demand the resignation of the former Prime Minister and his cabinet, revision of the constitution, and reformation of the monarchy in Thailand. The research of a M.A. student under my supervision investigated how different news agencies, one is a well-known Thai English-language news outlet another is an internally recognized global news agency, report news on this issue by using some discourse strategies to construct the representation of these Thai pro-democracy activists (Wongthai & Sawasdee, 2023). Interestingly, the results found that the international news agency constructs positive representation towards the activists, whereas the local news agency constructs the negative representation towards them. The results imply some interesting facts deep-rooted in Thailand. After learning this course, the knowledge gained reminds me that we can extend the current research by considering other modes besides writing in the news reports. The results of studying different modes may be different from the current research. Not only does the combination of many modes and a multimodal analysis can explain which mode has been specialized for what kinds of purposes, it also implies the hidden ideologies of news agencies, comparing with the study of only one mode. It would be more interesting to study multimodality in news reports in the intercultural aspect.
Reference
Wongthai, Nuntana & Sawasdee, Walairut. 2023. The representation of the new generation of Thai pro-democracy activists in online English-language news articles. LEARN Journal: Language Education and Acquisition Research Network, 16(2), 687-702. https://so04.tci-thaijo.org/index.php/LEARN/article/view/266979
My understanding of multimodal literacy feels limited after reading your deeply thought-out comment! Thanks for giving the context and making me see that while news in itself is an important form of realia for a language classroom, understanding its different modes and forms and acquainting learners with the analysis of these forms is a game changer altogether! Now I'm thinking that multimodal news reports can be leveraged limitlessly to engage the different cognitive abilites of learners!
your proposal to extend research into multimodal analysis in news reports, considering various modes and an intercultural perspective, is promising. It aligns well with the complexities of contemporary media and offers an opportunity to uncover hidden ideologies and understand the role of multimodal communication in shaping public discourse.
your proposal to extend research into multimodal analysis in news reports, considering various modes and an intercultural perspective, is promising. It aligns well with the complexities of contemporary media and offers an opportunity to uncover hidden ideologies and understand the role of multimodal communication in shaping public discourse.
I was intrigued by the fact that constructing meaning is subjective and influenced by various factors such as culture, personal biases, and emotional states, using different modes through various media. On top of that, in school, students are taught how to develop strong critical thinking skills, being open to diverse perspectives, and actively engaging with a variety of information sources can enhance their ability to construct meaningful understandings of the world around them. These facts brought me to my concern about the implementation of the new curriculum in my home country Indonesia.
Anyway, the term 'synaesthesia' is new to me. And I was wondering how this term differs from resemiotization (by Iedema, 2003). Looking forward for a discussion :)
Reference:
Iedema, R. (2003). Multimodality, resemiotization: extending the analysis of discourse as multi-semiotic practice. Visual Communication, 2(1), 29–57. https://doi.org/10.1177/1470357203002001751
Your ideas are very interesting. In my opinion I do not think that meaning-making process is subjective. We may influence by our own experience, communities we live, etc. but the way we make meanings out of various resources are not idiosyncratic. Signs we use are regular as others in the same community use over long periods of time as well as the way we make meanings.
For the term 'synaesthesis', I learn that it refers to meaning swapping, or transferring meaning from one mode to another. According to my understanding, it is like you can utter the sentence 'The sea is calm.' By communicating the same meaning, you can show the picture of the calm sea.
If the term 'resemiotization' means meaning-making that shift from one context to another, one group of people to another, one space to another, from practice to practice, or from one stage of practice to the next, I am not quite sure that it is the same thing.
Looking forward to a discussion on this as well. Thanks
Throughout the course of a typical day, one is invariably immersed in the multifaceted landscape of multimodal communication. This reality punctuates my personal experiences, underscoring the profound intricacy and diversity of contemporary communicative practices.
Consider the case of my mother, with whom I maintain regular telephonic correspondence. Our dialogues, conducted in our native Bengali, represent a distinct modality of communication. Simultaneously, interactions with my sister often materialize in the form of text messages that incorporate emojis and an amalgamation of languages, reflecting our multilingual abilities. These instances exemplify the complexity of modern communication, where language, digital media, and socio-cultural elements coalesce to form unique modes of expression.
Further exemplifying this multiplicity is my communication with my non-bengali girlfriend. Our interactions oscillate between textual exchanges and the sharing of images, culminating in the utilization of video chats. Each mode requires the deciphering of distinct semiotic codes, reinforcing the argument that contemporary communication necessitates a broad spectrum of interpretative skills that transcend traditional literacy.
Historically, literacy was confined to the ability to read and write text; however, such a definition seems ill-suited to the complex realities of the 21st-century communication landscape. The examples provided illustrate this point - the asynchronous and multilingual text communication with my sister, the visual literacy required to interpret images from my girlfriend, and the cultural and linguistic nuances involved in speaking with my mother. It becomes apparent that traditional literacy provides only a narrow view of the communication happening daily, primarily focusing on written forms, akin to the letter writing of my grandfather's era.
Hence, a more comprehensive approach, facilitated by multimodal analysis, seems necessary. Such an approach does not only consider the content (the what) but also places equal importance on the mode of communication (the how). It broadens the conceptual understanding of literacy by encompassing the myriad ways through which meaning is conveyed, acknowledged, and negotiated. This is in direct contrast to the conventional notion of literacy, which was largely mono-modal, centering on reading and writing as primary meaning-making activities.
By expanding the scope of literacy, the multimodal perspective offers a richer, more nuanced understanding of the wider field of literate practice. It acknowledges and embraces the diversity inherent in human communication, accommodating the multifaceted nature of communication modalities. Therefore, it fosters a more comprehensive, culturally responsive, and contextually relevant understanding of literacy in the digital age, equipping individuals to engage meaningfully within and beyond traditional literate environments.
Nat Turner stated, “Good communication is a way to separate between confusion and clarity.” One of the most crucial skills for us to have as students is multimodal communication. In essence, it is something we do virtually daily. I gained a much greater understanding of the various communication types during today's session. According to Liberator, Multimodal communication is a phrase used to refer to all the various methods we use on a daily basis to communicate with one another. This can be done verbally, through messaging, tweeting, emailing, scribbling, gesturing, and body language, as well as with the use of a communication device. We utilize and observe multimodal communication almost daily, whether it be in official, professional, informal, or ordinary contexts, making multimodal analysis useful. It is about having more types of communication modes. In today's generation, technology has been taking over our everyday lives according to statistics, more than five billion people used the internet as of April 2022, making up 63.1 percent of the world's population. Out of this total, 4.7 billion people, or 59% of the global population, used social media. . Compared to the traditional notion of literacy, its purpose is still the same. However, due to innovation, we are expected to do more and adapt to the new technology, resulting in a combined or multi-mode of communication. We are already living in a more advanced world. Based on my experience with multimodal communication, it helped me express myself more. For example, on some social media platforms like Facebook, Instagram, and more, I can create and share my ideas visually and by text. One of its advantages, as well, is by means of learning. Having text, graphics, and visual tools makes learning easier and more fun. I can say that this transformation in communication has been a great help in our society. The process of developing ideas has become less complicated. Currently, I am taking a course that is somewhat related to design and communication, and I realized that due to this innovation, different job opportunities arise. One of them is my dream job as a multimedia artist, and I believe that this topic has been a great help in adding core knowledge that I will need in the future. Reading and writing were the only forms of literacy in the past, but as time has gone on, literacy has come to mean many various things. The use of literacy in today's age is not part of a larger picture where it is still employed in even the most fundamental parts of life.
Reference:
https://www.linkedin.com/pulse/good-communication-bridge-between-confusion-clarity-prakash-rao#:~:text=%22Good%20communication%20is%20the%20bridge,clarity.%22%20%2D%20Nat%20Turner.
https://www.statista.com/statistics/617136/digital-population-worldwide/
https://liberator.net.au/support/education/aac/multi-modal#:~:text=Multi%20Modal%20Communication%20is%20simply,by%20using%20a%20communication%20device.
I am an early years primary teacher and a site of multimodal communication I use on a daily basis is Seesaw. Seesaw is an online platform that allows parents and teachers to view the child's learning and educational experience through multimodal literacies such as videos, pictures, PowerPoint presentations, vocal messages and so forth. A platform like Seesaw becomes a facilitator for learning for multiple reasons:
1. The platform allows teachers and parents to 'work together, work is posted on a daily basis and a communication bridge is established between the two parts. Plenty of research in fact shows that when 'learning in tandem' ( Newland and Metternes, 1996) takes place, the outcomes on the child's learning are greater
2. As Feubert observed, young children learn through play. An online platform such as Seesaw, allows recording of the child's learning through play. For example, I assigned a task in which children were 'shapes detectives' and had to notice shapes in an outside environment. Whilst some parents, decided to upload pictures and audio recordings of a child naming shapes, others filmed children walking around their home naming shapes in the environment. Until not long ago, children would have been asked to make a model of something and bring it into school to show the teacher and peers 'proof of their learning. This traditional approach to home learning implied that children had means and materials at home to complete the task, many parents completed the task themselves for a matter of time consumption as well as the fact they wanted the work to look good. But now, the use of multimodal literacies allows learning to happen ( and be assessed ) in a more pedagogical-coherent and child-friendly way.
3. Because of the possibility to pick and choose the mode, children play an even more active role when producing meaning and are easily more reconnected to their synaesthetic experience of the world. So, even though the task is set by myself ( which already constitutes a limitation of the child's experience when engaging with the environment), the child has the agency to explore and interpret his surrounding using the mode that best suits him.
However, being able to use multimodal approaches implies 1. teachers being trained to integrate various modes in their teaching. Mainstream schools in the UK, unfortunately, suffer from constant cuts to budgets which limits teachers' professional developments as well as not providing the technology necessary for children to actually engage in multimodal literacies. Schools are also tied to an inspection system that limits the personal agency of a teacher, expecting the National Curriculum to be delivered in a more traditional way.
2.On the other side, Children and parents need to have access to the knowledge and the technology necessary to make mode-switching happening.