AI multimodal feedback produces post-test learning gains equivalent to educator feedback after controlling for pre-test performance¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r2 · q3 · i2 medium
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? In a randomized online experiment, post-test scores did not differ significantly between the AI multimodal feedback condition and the business-as-usual educator feedback condition after controlling for pre-test scores. → Chloe Qianhui Zhao 2026 q3 i2 Both conditions improved significantly from pre-test to post-test, with within-group effect sizes of d = 0.5376 (educator feedback) and d = 0.7623 (AI feedback). → Chloe Qianhui Zhao 2026 (2)

Evidence¶

Chloe Qianhui Zhao 2026¶

Chloe Qianhui Zhao, Jie Cao, Jionghao Lin, and Kenneth R. Koedinger. (2026). LLM-based Multimodal Feedback Produces Equivalent Learning and Better Student Perceptions than Educator Feedback. LAK 2026. https://arxiv.org/abs/2601.15280 doi:10.48550/arxiv.2601.15280

q3 · i? · causal · r2

ANCOVA in the RQ1.1 learning evaluation, with post-test score as dependent variable, condition as between-group factor, and pre-test as covariate; homogeneity of regression slopes was satisfied (p = 0.24). The "effect of condition on post-test scores was not significant" with partial η² = 0.0047.

"After controlling for pre-test performance, the effect of condition on post-test scores was not significant (𝐹=0.91,𝑝=0.34, partial𝜂 2 =0.0047)."

Chloe Qianhui Zhao 2026 (2)¶

Chloe Qianhui Zhao, Jie Cao, Jionghao Lin, and Kenneth R. Koedinger. (2026). LLM-based Multimodal Feedback Produces Equivalent Learning and Better Student Perceptions than Educator Feedback. LAK 2026. https://arxiv.org/abs/2601.15280

q3 · i2 · causal · r2

Paired-samples t-tests within each condition in the RQ1.1 analysis of the randomized experiment (197 learners). The AI group improved significantly with Cohen's d = 0.7623; the baseline group's paired test gave d = 0.5376 (t = −3.4841, p = 0.0006).

"The AI Multimodal Feedback group also demonstrated significant improvement from pre- to post-test (𝑡=− 5.6273, 𝑝<0.001***,𝐶𝑜ℎ𝑒𝑛′𝑠𝑑=0.7623)."

Discussion¶

Related Claims¶

• Feedback type shifts engagement by question type: educator feedback yields more MCQ submissions while AI feedback trends toward more open-ended revisions, with no overall difference — related
• Learners receiving ChatGPT-generated hints showed statistically significant learning gains compared to a no-hint control in an online mathematics tutoring experiment — related
• Non-layered AI feedback produced significantly higher post-test performance than layered AI feedback after controlling for pre-test — related
• Project performance improved significantly from pre- to post-feedback across all three feedback conditions with no significant differences between feedback sources — related
• No significant group differences in knowledge gain or knowledge transfer despite ChatGPT's task-performance advantage — related
• Students show large immediate pre-to-post knowledge gains within a single LLM-supported learning session — related
• Learning gains from ChatGPT-generated hints were not statistically significantly different from human tutor-authored hints, though ChatGPT gains were numerically higher in all subjects — related
• Post-test mathematics performance was significantly lower than pre-test after AI-tutor use — related