Exploratory ANCOVA controlling for mastery probability favored the adaptive condition, but is not interpretable as a causal treatment effect¶

Claim · All claims
Evidence · 1 study · causal r3 · q3 · i3 large
Currency · newest study 2026 · checked against its sources 2026-10-19 (automated, 1 of 1 entry) · not reviewed by a person

Subclaims¶

q3 i3 When mastery probability was added as a covariate, the adaptive condition showed higher adjusted posttest performance (d = 0.46), though the authors caution against causal interpretation. → Sibley 2026

Evidence¶

Sibley 2026¶

Sibley, L., Berner, T., & Schmalfeldt, T. (2026). Effectiveness of adaptive versus non-adaptive intelligent tutoring systems in early primary mathematics. Computers and Education Open 11. https://doi.org/10.1016/j.caeo.2026.100420

q3 · i3 · causal · r3

Exploratory ANCOVA of posttest on condition with prior knowledge and mastery probability as covariates in the N = 132 experiment; mastery probability itself significantly predicted posttest (F(1, 122) = 51.83, p < .001, η² = .30). The article prints d = 0.46 for the adjusted condition difference.

"Adjusted marginal means indicated higher posttest performance in the adaptive condition (M = 0.12, SE = 0.07) than in the non-adaptive condition (M = −0.12, SE = 0.07), t(122) = 2.33, p = .022, d = 0.46."

Discussion¶

Related Claims¶

• Intelligent tutoring systems improve achievement over non-individualised instruction — the general claim this finding bears on, with the studies behind it
• Objective learning gains did not differ significantly by condition in the primary ANCOVA, but KBD showed a significant positive effect on learning gains in exploratory regressions accounting for learner covariates — related
• Adaptive spaced retrieval practice produced higher end-of-semester posttest performance than learner-directed AI study, but fixed spaced retrieval did not significantly outperform learner-directed study — related
• Students in the non-adaptive condition reached significantly higher average mastery probabilities than students in the adaptive condition — related
• Adaptive task difficulty in an ITS did not improve second-graders' mathematics posttest performance compared with an identical non-adaptive ITS — reports the opposite