Adaptive task difficulty in an ITS did not improve second-graders' mathematics posttest performance compared with an identical non-adaptive ITS¶

Claim · All claims
Evidence · 1 study · causal r3 · q3 · i0 negligible
Currency · newest study 2026 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person

Subclaims¶

q3 i0 In a randomized two-group pretest–posttest field experiment (N = 132), the adaptive ITS condition did not outperform the non-adaptive condition on posttest mathematics scores controlling for prior knowledge. → Sibley 2026

Evidence¶

Sibley 2026¶

Sibley, L., Berner, T., & Schmalfeldt, T. (2026). Effectiveness of adaptive versus non-adaptive intelligent tutoring systems in early primary mathematics. Computers and Education Open 11. https://doi.org/10.1016/j.caeo.2026.100420

q3 · i0 · causal · r3

Randomized two-group pretest–posttest field experiment with 132 second-graders; ANCOVA of posttest on condition controlling for prior knowledge found no condition effect. The article reports "no significant effect of condition, F(1, 124) = 0.32, p = .574, η²p = .00"; the printed η²p = .00 is a negligible effect: no difference was detected in this sample, which does not show that the true effect is exactly zero.

"However, there was no significant effect of condition, F(1, 124) = 0.32, p = .574, η²p = .00, indicating that – against our assumption – the adaptive version of the ITS was not more beneficial for students regarding their learning outcome than a non- adaptive ITS."

Discussion¶

Related Claims¶

• Intelligent tutoring systems improve achievement over non-individualised instruction — the general claim this finding bears on, with the studies behind it
• Adaptive scaffolding policies (BKT and DRL) significantly improve posttest performance over a non-adaptive control in a logic ITS — reports the opposite
• All training policies narrowed the prior-knowledge achievement gap from pretest to posttest, with BKT achieving the largest reduction (77.1%) — related
• The Expert Mathematician shows a positive but not statistically significant effect on middle school math achievement (effect size 0.35, improvement index +14 percentile points) — related
• Latent learner profiles did not moderate the effectiveness of the adaptive ITS — related
• Exploratory ANCOVA controlling for mastery probability favored the adaptive condition, but is not interpretable as a causal treatment effect — reports the opposite