Metacognitive and affective chatbot feedback each produce higher retention scores than neutral feedback, with no significant difference between metacognitive and affective groups¶

Claim · All claims
Evidence · 1 study · causal r2 · q3 · i2 medium
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i2 After controlling pre-test knowledge, retention scores differed significantly by feedback type (F = 7.51, p = 0.001, effect size 0.15), with metacognitive and affective feedback each higher than neutral feedback. → Yin 2025

Evidence¶

Yin 2025¶

Yin, J., Xu, H., Pan, Y., & Hu, Y. (2025). Effects of different AI-driven Chatbot feedback on learning outcomes and brain activity. npj Science of Learning. https://doi.org/10.1038/s41539-025-00311-8

q3 · i2 · causal · r2

Randomized experiment in which 87 college students (29 per group after exclusions) interacted with biology chatbots giving metacognitive, affective, or neutral feedback. ANCOVA showed "a signi ficant difference in the feedback type ( F = 7.51, p = 0.001)" with effect size 0.15 and power 0.94. Post-hoc tests showed MF and AF each higher than NF, but MF and AF comparable (corrected p = 0.580).

"After controlling the influence of pre-test knowledge test scores, students’retention scores showed a signi ficant difference in the feedback type ( F = 7.51, p = 0.001). The effect size was 0.15, and the statistical power was 0.94."

Discussion¶

Related Claims¶

• Metacognitive chatbot feedback produces higher transfer scores than both affective and neutral feedback — related
• Metacognitive chatbot feedback raises metacognitive sensitivity relative to affective and neutral feedback — related
• Feedback types elicit distinct fNIRS activation patterns: frontopolar and middle temporal gyrus higher under metacognitive feedback, supramarginal gyrus under affective feedback, DLPFC under neutral feedback — related
• Using the chatbot-based journaling system improves self-reported SRL strategy use from pre-survey to 12-week follow-up in all groups, including baseline — related
• Experimental and control groups did not differ significantly on pre-test NOS or SK scores — a broader claim this one bears on