Non-layered AI feedback produced significantly higher post-test performance than layered AI feedback after controlling for pre-test¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r2 · q3
Currency · newest study 2026 · checked against its sources 2026-10-20 (automated, 2 of 2 entries) · not reviewed by a person

Subclaims¶

q3 i? Students receiving non-layered elaborated AI feedback scored significantly higher on the post-test than students receiving layered feedback (adjusted means 11.24 vs 10.40). → Jie Cao 2026

Evidence¶

Jie Cao 2026¶

Jie Cao, Chloe Qianhui Zhao, Christian Schunn, Elizabeth A. McLaughlin, Jionghao Lin, Kenneth R. Koedinger. (2026). Assessing the Impact and Underlying Pathways of Sequenced AI Feedback on Student Learning. arXiv. https://arxiv.org/abs/2604.07469 doi:10.48550/arxiv.2604.07469

q3 · i? · causal · r2

Randomized experiment (layered n=100, non-layered n=99) in an online higher-education lesson on the multimedia principle; ANCOVA on post-test with pre-test as covariate. The contrast was significant, t(196) = 2.32, p = .02, and the model accounted for 34% of variance (R2 = .34).

"After controlling for initial performance, students in the non-layered feed- backgroupachievedsignificantlyhigherpost-testscores(M ad j = 11.24,SE= 0.25) than those in the layered feedback group (Mad j = 10.40,SE= 0.25)."

Jie Cao 2026 (2)¶

Jie Cao, Chloe Qianhui Zhao, Christian Schunn, Elizabeth A. McLaughlin, Jionghao Lin, Kenneth R. Koedinger. (2026). Assessing the Impact and Underlying Pathways of Sequenced AI Feedback on Student Learning. arXiv. https://arxiv.org/abs/2604.07469

q3 · i? · causal · r2

The same ANCOVA model testing feedback condition with pre-test covariate was statistically significant, F(2,196) = 50.39, p < .001, with R2 = .34; Table 1 reports η2 = 0.027 for the condition effect.

"The model was statistically significant, F(2,196) = 50.39,p < .001, accounting for 34% of the variance (R2 =.34)."

Discussion¶

Related Claims¶

• Layered AI feedback was perceived as more encouraging and independence-supporting but imposed higher mental effort — related
• The total effect of layered feedback on learning outcomes was significantly negative, with an unmediated negative direct effect — related
• AI multimodal feedback produces post-test learning gains equivalent to educator feedback after controlling for pre-test performance — related
• In a 16-week quasi-experiment, A-Level STEM students receiving high-frequency automated marking scored 16.8 percentage points higher on a blind-graded final mock exam than control students with human marking — related
• Senior high school students taught with generative AI-supported instruction scored significantly higher on geometric reasoning and proof construction than students taught conventionally — related
• The post-test advantage of generative AI-supported instruction persists after controlling for pre-test performance (ANCOVA) — related
• Adaptive spaced retrieval practice produced higher end-of-semester posttest performance than learner-directed AI study, but fixed spaced retrieval did not significantly outperform learner-directed study — related
• In a controlled evaluation with 268 ninth-grade students, embedding elaborated feedback in the ACED system did not impair assessment quality and produced significantly greater learning than a control group — related
• TAIA produced higher adjusted post-test scientific achievement than worksheet-supported inquiry, controlling for pre-test and ICT competency — related