Pedagogical augmentations of ITSs show inconsistent effects: self-assessment helped in some studies but not others, while personalization and matched test-practice format showed benefits¶

Claim · All claims
Evidence · 1 study (2 entries) · review r3 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Among ITS-vs-modified-ITS studies, personalization improved performance, progress information showed a medium-to-large effect in one study, but metacognitive supports (self-assessment, performance threshold) showed inconsistent effects across studies. → Létourneau 2025 q2 i? Learning gains were twice as high when the test format matched the practice environment compared to when students had to transfer knowledge across formats. → Létourneau 2025 (2)

Evidence¶

Létourneau 2025¶

Létourneau, A., Martineau, M. D., Charland, P., Karran, J. A., Boasen, J. & Léger, P. M. (2025). A systematic review of AI-driven intelligent tutoring systems (ITS) in K-12 education. npj Science of Learning 10:29. https://doi.org/10.1038/s41539-025-00320-7

q2 · i? · review · r3

The review's synthesis of eleven ITS-vs-modified-ITS studies: only three reported effect sizes, mostly small. McCarthy et al.'s 2x2 iSTART study found no condition effects, contradicting Long and Aleven's self-assessment findings. Walkington and Bernacki's personalization benefits were limited to students highly engaged with their interests.

"Contrary to the three previous studies ’ results which suggested that self-assessment enhanced the effectiveness of the ITS44,46, none of the experimental conditions in this study were reported to influence performance33."

Létourneau 2025 (2)¶

Létourneau, A., Martineau, M. D., Charland, P., Karran, J. A., Boasen, J. & Léger, P. M. (2025). A systematic review of AI-driven intelligent tutoring systems (ITS) in K-12 education. npj Science of Learning 10:29. https://doi.org/10.1038/s41539-025-00320-7

q2 · i? · review · r3

Borchers et al.'s quasi-experiment with 82 ninth-graders using Mathtutor found a significant main effect of matching practice and test format (t(567.96) = -3.28, p = 0.001), though overall ITS vs paper learning gains were similar.

"learning gains were twice as high when the test format matched the practice environment (M = 0.18, SD = 0.30) compared to when there was no such match and students had to transfer knowledge across formats (M = 0.10, SD = 0.30)"

Discussion¶

Related Claims¶

• The classroom testing effect transfers to mismatched test formats, but consistent quiz–exam formats yield a significantly larger effect (g = 0.531 vs. g = 0.399). — related
• Students with lower baseline performance (below the 50th percentile) showed significantly greater math achievement gains with ASSISTments than matched virtual peers — related
• Effects of ITSs are mitigated when compared to non-intelligent tutoring systems: three of four comparisons found no significant difference in learning gains — related
• Most ITS-versus-teacher studies in K-12 report a positive effect of ITSs on student performance, with medium-to-large effect sizes — related