Over the past decades, socio-constructivist theories have emphasized the high relevance of social interactions for learning. Researchers explain the efficiency of face-2-face learning formats due to the importance of social interactions for learnings.

• Toddlers acquire language by interacting with their care-givers
• students learn from their peers and teachers
• adults pursue life-learning through their communities of practice
• citizens working in teams are more likely to find solutions to global issues (e.g., climate change) compared to individuals alone.

Relevance of social interactions for learning; Source: Ochoa, X. (2016): Multimodal learning analytics

In the course videos we were introduced to new ways of representing knowledge in the context of multimodal learning. Given to new digital learning formats and new technologies arising, researchers aim to find out whether assessing and acquiring collaborative learning skills can be transformed through new multimodal sensing technologies.

What is multimodal learning analytics?

Definition learning analytics; Source: Ochoa, X. (2016): Multimodal learning analytics

Ochoa relates the topic of multimodal learning analytics to the following new fields of research:

• Behaviorism
• Cognitive Science
• Multimodal Interaction (HCI)
• Computer Vision
• Natural Language Processing
• Biosignals Processing

How does multimodal learning analytics work?

Over the past decade, high frequency sensors (such as eye-trackers, motion sensors, wearables) have become affordable and reliable and educational researchers can now collect significantly larger datasets. These new conditions are opening new doors for capturing students’ behavior. As a consequence, the field of multimodal learning analytics allows researchers to tackle new research questions and develop new educational interventions related to the learning process.

New technologies to analyze learners‘ behavior; Source: Learning, Innovation and Technology Lab, Harvard University

Video - new technologies to analyze learners‘ behavior; Source: Learning, Innovation and Technology Lab, Harvard University

The Microsoft Kinect sensor for example, collects information about a person’s body joints and helps to describe human movement using non-verbal characteristics. possibilities of combining sensors with data mining techniques.

Description of human movements through Microsoft Kinect sensor; Source: Ochoa, X. (2016): Multimodal learning analytics

What are the challenges?

"We should be looking where it is useful to look, not where it is easy" - Ochoa, X. (2016)

Streetlight effect – multimodal learning analytics; Source: Ochoa, X. (2016): Multimodal learning analytics

According to Ochoa, the field of multimodal learning analytics is an open but still dark field since we don’t know which modes are important to understand the learning process and what are the relevant features of those signals.

Possibilities of signals affecting a learning process (e.g. what we see, what we hear, how we move, our pulse, brain activity and more); Source: Ochoa, X. (2016): Multimodal learning analytics

What’s next in multimodal learning analytics?

New high-frequency data collection technologies and machine learning analysis techniques could offer new insights into students' learning trajectories. Today, most researches related to multimodal learning analytics focus on online courses or cognitive tutors, in which the tasks are more structured and the entirety of interaction happens in front of a computer. (Ochoa, X. (2016))

Multimodal learning analytics could offer new insights into students' learning trajectories. (Blikstein, P. (2013)). However, researchers are still at the beginning and there is a lot of exploring to do. Today, several researchers aim to build a mode integration framework for multimodal learning analytics. ((Ochoa, X. (2016)). This includes for example the development of reliable multimodal measuring devices.

A personal note

As data scientist working in the field of human resource development, the topic of learning analytics is of high relevance for my daily work. I am very excited to see how learning analytics will be applied in the future to better understand learning behavior of workers. Multimodal learning analytics can help us to learn more about the efficiency of multimodal learning approaches and to derive best practices for learning offers. I am excited to see what’s next in the field of multimodal learning analytics :-)

Sources:

Ochoa, X. (2016): Multimodal learning analytics, URL: https://www.slideshare.net/xaoch/multimodal-learning-analytics-60955642?from_action=save

Learning, Innovation and Technology Lab, Harvard University, URL: https://lit.gse.harvard.edu/multimodal-learning-analytics

Blikstein, P. (2013), Multimodal learning analytics, URL: https://www.researchgate.net/publication/266653654_Multimodal_learning_analytics

Further readings:

Microsoft Kinect sensor: https://developer.microsoft.com/en-us/windows/kinect/

Ferguson, R; Clow, D. (2015): Examining engagement: analysing learner subpopulations in massive open online courses (MOOCs), URL: https://dl.acm.org/doi/10.1145/2723576.2723606

Davies, R.; Nyland, R.; Chapman, J. (2015): , Using transaction-level data to diagnose knowledge gaps and misconceptions, URL: https://dl.acm.org/doi/10.1145/2723576.2723620