Most ITS-versus-teacher studies in K-12 report a positive effect of ITSs on student performance, with medium-to-large effect sizes¶

Claim · All claims
Evidence · 1 study · review r3 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Seven of eight studies comparing an ITS to traditional teaching reported a positive effect in favor of the ITS, with medium to large effect sizes. → Létourneau 2025

Evidence¶

Létourneau 2025¶

Létourneau, A., Martineau, M. D., Charland, P., Karran, J. A., Boasen, J. & Léger, P. M. (2025). A systematic review of AI-driven intelligent tutoring systems (ITS) in K-12 education. npj Science of Learning 10:29. https://doi.org/10.1038/s41539-025-00320-7

q2 · i? · review · r3

The review's discussion of the ITS-vs-teacher category found "seven out of eight articles reported a positive effect in favor of the ITS," though the review declined to tabulate recalculated effect sizes because designs varied too much across studies.

"In comparison between an ITS and teacher-led instruction, seven out of eight articles reported a positive effect in favor of the ITS, with a medium to large size effect."

Discussion¶

Related Claims¶

• Effects of ITSs are mitigated when compared to non-intelligent tutoring systems: three of four comparisons found no significant difference in learning gains — related
• In the grades 4-5 cluster RCT, ITSS produced statistically significant positive effects on all six comprehension outcomes (average effect size 0.24) — a narrower finding that bears on this claim
• In the grade 7 cluster RCT, ITSS produced statistically significant positive effects on all eight comprehension outcomes (average effect size 0.23) — a narrower finding that bears on this claim
• Pedagogical augmentations of ITSs show inconsistent effects: self-assessment helped in some studies but not others, while personalization and matched test-practice format showed benefits — related
• The introduction reports that the VCR tutor produced statistically significant performance gains in using the VCR, a rare result for ITSs at the time — a narrower finding that bears on this claim