Certain multimodal data combinations improve predictive model performance, with audio plus eye-tracking data most effective in one K–8 study¶

Claim · All claims
Evidence · 1 study · review r2 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person · recheck by 2028: tool and model accuracy changes quickly

Subclaims¶

q2 i? Combining modalities enhanced prediction of learning gains and post-test scores, with audio and eye-tracking data the most effective combination. → Caskurlu 2025

Evidence¶

Caskurlu 2025¶

Caskurlu, S., Ocak, C., & Dai, C.-P. (2025). The Scope of Multimodal Learning Analytics in K–8: A Systematic Review. Journal of Learning Analytics, 12(2), 224–236. https://doi.org/10.18608/jla.2025.8505

q2 · i? · review · r2

The review's discussion reports this as a key finding from the included studies: "certain combinations of modalities provide better model performance, depending on the context MMD collected from." The review attributes the finding to Olsen et al. (2020), comparing named modality combinations.

"Olsen et al. (2020) found that integrating audio and eye-tracking data was the most effective combination for prediction, among the combinations of audio-gaze-dialogue, audio-gaze-log, and audio-log."

Discussion¶

Related Claims¶

• Facial features of negative emotions predict low rapport, but additional modalities added no predictive power — related
• Multimodal data can reverse the interpretation of silence in collaborative problem-solving — related
• Combining more modalities does not always improve detection of learner mental states — related
• AIOL research shows increasing emphasis on multimodal data integration for emotion recognition and personalized learning support — related
• A head-confined multimodal configuration (EEG, eye tracking, forehead EDA) predicts held-out learners' five-level engagement ratings at 75.0% balanced 1-off accuracy, 12.0 points above the mode baseline — a narrower finding that bears on this claim
• Multimodal behavioral profiles of dyads' collaborative learning are identifiable by integrating video, audio, and log data, with emergent processes more explanatory than individual behaviors
• Only a minority of teacher educators interpreted a critical classroom-management incident as theoretically expected, despite marked eye-tracking differences — related
• AI-powered multimodal approaches tracking emotional and physiological responses increase student engagement and motivation in collaborative learning — related
• Chest-worn ECG provides the strongest cardiac-sensing performance for attention-difficulty prediction, and fusing all cardiac streams does not improve upon chest ECG alone — related
• Combining eye tracking with other process measures such as machine vision features provides useful information about student engagement — related