ITS evaluations in K-12 predominantly use short quasi-experimental designs concentrated in STEM, raising novelty-effect concerns¶

Claim · All claims
Evidence · 1 study · review r2 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Researchers mainly used quasi-experimental methods, 82% of studies were in STEM subjects, and half of interventions lasted less than a week, making novelty effects an unaddressed threat. → Létourneau 2025

Evidence¶

Létourneau 2025¶

Létourneau, A., Martineau, M. D., Charland, P., Karran, J. A., Boasen, J. & Léger, P. M. (2025). A systematic review of AI-driven intelligent tutoring systems (ITS) in K-12 education. npj Science of Learning 10:29. https://doi.org/10.1038/s41539-025-00320-7

q2 · i? · review · r2

The review's analysis of the 28 included studies (Fig. 1) found quasi-experimental pre/post designs dominated, most studies were in STEM (82%), and interventions were often very short. The review notes such short durations may amplify effect sizes via novelty.

"half of the interventions lasted less than a week28,30,32,33,36,39,41,42,44,46,47,51, with some as brief as a single class period35,46,51"

Discussion¶

Related Claims¶

• Most AI-motivation studies lasted less than 10 weeks, raising novelty-effect concerns and leaving long-term effects unexamined — related
• RL-based educational applications concentrate on STEM subjects (n = 39 of 62 papers specifying a subject) and on college/higher-education learners (n = 30), with 36 papers not specifying a target group — related
• AITS learning effectiveness evaluated via pre-test/post-test experimental/control group study showed very encouraging results — related