Facial features of negative emotions predict low rapport, but additional modalities added no predictive power¶

Claim · All claims
Evidence · 1 study · review r2 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person · recheck by 2028: tool and model accuracy changes quickly

Subclaims¶

q2 i? In one reviewed study, facial features related to negative emotions were associated with low rapport, but information from other modalities did not further improve predictive power. → Rachelle Esterhazy 2025

Evidence¶

Rachelle Esterhazy 2025¶

Rachelle Esterhazy, Rogers Kaliisa, Daniel Sanchez, Malcolm Langford and Crina Damsa. (2025). Advancing Multimodal Collaboration Analytics: A Scoping Review. Journal of Learning Analytics 12(2). https://doi.org/10.18608/jla.2025.8625

q2 · i? · review · r2

The review reports, citing Müller et al. (2018), a multimodal low-rapport detection approach based on audio and video data including facial expressions, hand motion, speech activities, and prosodic features. "certain facial features related to negative emotions (e.g., nose wrinkling, brow-raising) are also associated with low rapport". No effect sizes are printed.

"They find that certain facial features related to negative emotions (e.g., nose wrinkling, brow-raising) are also associated with low rapport but that additional information from the other modalities did not further improve predictive power in this regard."

Discussion¶

Related Claims¶

• Ablation of feature-vector models: time-window features add no predictive power to logistic regression but boost a feedforward network, and total count features substantially boost performance on all datasets — related
• Multimodal data can reverse the interpretation of silence in collaborative problem-solving — related
• Certain multimodal data combinations improve predictive model performance, with audio plus eye-tracking data most effective in one K–8 study — related
• AI-powered multimodal approaches tracking emotional and physiological responses increase student engagement and motivation in collaborative learning — related
• SHAP modality attributions aligned with each BARS dimension's behavioral focus, supporting interpretable feedback — related
• In a Chicago middle-grades study, the combination of grades and attendance provided the best prediction of high school on-track rates, and adding test scores and grit measures did not further improve prediction — related
• Combining eye tracking with other process measures such as machine vision features provides useful information about student engagement — related