ITS produce significant positive effects at all grade levels and in almost all subject domains evaluated, with exceptions such as chemistry¶

Claim · All claims
Evidence · 1 study (2 entries) · quant-synthesis r2–r3 · q4 · i0 negligible
Currency · newest study 2014 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q4 i? The use of ITS produced moderate, statistically significant mean effect sizes at all grade levels under both fixed and random-effects models. → Wenting Ma 2014 q4 i0 Significant positive effects appeared in most subject domains, but for chemistry as well as literacy and language learning the mean effect sizes were small (g = .16) and moderate (g = .34), with chemistry not significant under random effects. → Wenting Ma 2014 (2)

Evidence¶

Wenting Ma 2014¶

Wenting Ma, Olusola O. Adesope, John C. Nesbit, and Qing Liu. (2014). Intelligent Tutoring Systems and Learning Outcomes: A Meta-Analysis. Journal of Educational Psychology. https://doi.org/10.1037/a0037123

q4 · i? · quant-synthesis · r3

Grade-level moderator analysis (Table 3) covering elementary, middle school, high school, postsecondary, and mixed-grade studies; the between-levels difference was not statistically significant under a random-effects model (QB p = .82).

"T h e u s e o f IT S p r o d u c e d m o d e ra te s ta tis tic a lly s ig n if ic a n t m e a n e ff e c t siz e s a t a ll g r a d e le v e ls u n d e r b o th th e f ix e d a n d r a n d o m - e f f e c t s m o d e ls ."

Wenting Ma 2014 (2)¶

Wenting Ma, Olusola O. Adesope, John C. Nesbit, and Qing Liu. (2014). Intelligent Tutoring Systems and Learning Outcomes: A Meta-Analysis. Journal of Educational Psychology. https://doi.org/10.1037/a0037123

q4 · i0 · quant-synthesis · r2

Subject-domain moderator analysis (Table 3) with heterogeneity QB = 50.67, p < .001 under the fixed-effect model. Chemistry (g = .16) and language and literacy (g = .34) were the weaker domains; humanities and social science (g = .63) and biology and physiology (g = .59) showed the largest random-effects means.

"For chem istry a s well as literacy and language learning, the use of ITS prod uced sm all and m oderate m ean effect sizes (g = .16 and g = .34, re spectively)."

Discussion¶

Related Claims¶

• ITS effectiveness is not moderated by whether the ITS provides feedback or models student misconceptions — related
• AI-supported instruction has a moderate positive effect on student achievement in STEM education (g = 0.670 under a random effects model) — a broader claim this one bears on
• Bayesian network modeling ITS showed a higher weighted mean effect size than model tracing and constraint-based modeling, but only under a fixed-effect model — related
• GenAI chemistry outputs can be fluent and locally persuasive while chemically invalid across representations — related
• AR/VR tools show the largest effect among AI application types (g = 1.765), while ITS and personalized educational games show small effects — related
• LLMs struggle most with Chemistry riddles, with best EM accuracy at or below 75% versus above 90% in other subjects — related
• School level significantly moderates effects: AI-enabled personalized STEM education is more effective in junior and senior high school than in kindergarten — related
• AI use is predominantly independent of the science subject taught, with one exception: chemistry teachers most often cite lack of subject-specific relevance — related