Definitional framework of five modalities in MMCA research¶

Theory · All theories
Evidence · 1 claim (1 for) · 1 study (1 review), q2 · 0 of 1 report an effect size · 1 claim rests on one study

Description¶

The review develops a definitional framework classifying modalities used in MMCA studies into five categories: auditory, visual, physiological, kinesthetic, and tactile. It defines modality as "the distinct sensory channels or modes through which humans (or machines) perceive and interact with the external environment", grounding the framework in relationships between sensors and the stimuli they detect. Each modality is linked to the information it can extract about learners, such as prosodic and semantic speech, position and movement, gaze, facial expressions, actions, written output, and physiological arousal.

Design Implications¶

Context¶

Requirements¶

• Studies must use more than one modality in data collection and analysis to fall within the framework's scope

Constraints¶

• The framework was derived from 36 empirical studies reviewed up to 31 January 2023; no studies using olfactory or gustatory stimuli were found in the reviewed literature

Target Learners¶

• learners in higher education, secondary education, and primary education collaborative settings

Target Learning Objectives¶

• understanding collaborative learning processes and outcomes through multimodal data

Claims¶

• Combining more modalities does not always improve detection of learner mental states [+W]

Related Theories¶

• Four rationales underlie combining modalities in MMCA research
• Multimodal analysis: studying learning with multiple integrated data streams

Examples¶

• Triangulate physiological data with other CSCL data streams ( such as video or logfile traces ) to uncover otherwise unidentifiable psychophysiological reactions

Key Sources¶

• Rachelle Esterhazy, Rogers Kaliisa, Daniel Sanchez, Malcolm Langford and Crina Damsa. (2025). Advancing Multimodal Collaboration Analytics: A Scoping Review. Journal of Learning Analytics 12(2). https://doi.org/10.18608/jla.2025.8625