Ma et al. (2014) meta-analysis: step-based ITSs outperformed no tutoring and large-group teacher-led instruction, but showed no differences against small-group or one-to-one human tutoring¶

Claim · All claims
Evidence · 1 study · review r2 · q2
Currency · newest study 2015 · checked against its sources 2026-10-31 (automated, 1 of 1 entry) · not reviewed by a person

Subclaims¶

q2 i? ITSs showed positive effects versus no tutoring and large-group teacher-led instruction, but no differences versus small-group or one-to-one human tutoring. → Benedict du Boulay 2015

Evidence¶

Benedict du Boulay 2015¶

Benedict du Boulay. (2015). Recent Meta-reviews and Meta-analyses of AIED Systems. Int J Artif Intell Educ. https://doi.org/10.1007/s40593-015-0060-1

q2 · i? · review · r2

The letter reports the Ma et al. (2014) meta-analysis of step-based ITSs: "positive results for step-based ITSs" against no tutoring and large-group teacher-led instruction, with no differences against small-group or one-to-one tutoring. No effect sizes are printed.

"Ma et al. (2014) found similar positive results for step-based ITSs both when compared to a no tutoring condition and when compared to large group human teacher led-instruction, but no differences when compared to small group human tutoring or to one –to–one tutoring"

Discussion¶

Related Claims¶

• Kulik and Fletcher (2015) found similar-sized ITS improvements to other meta-analyses, with larger effect sizes when tests were tuned to the tutoring system than when standardised tests were used — related
• ITS significantly outperform large-group human instruction, individual computer-based instruction, and textbooks or workbooks, but not small-group or individual human tutoring — possibly the same claim (merge candidate)
• Meta-analytic evidence indicates intelligent tutoring systems yield positive learning outcomes compared with non-ITS conditions — a broader claim this one bears on
• Intelligent tutoring systems improve achievement over non-individualised instruction — possibly the same claim (merge candidate)
• ITS substitution: meta-analyses find no significant cognitive learning differences between ITS and human tutoring or small-group instruction — possibly the same claim (merge candidate)
• Learning with intelligent tutoring systems yields a moderate, statistically significant overall advantage in achievement over non-ITS modes of instruction — a broader claim this one bears on
• Nesbit et al. (2014) found a significant advantage of ITSs over teacher-led classroom instruction and non-ITS computer-based instruction in teaching programming — related
• VanLehn (2011) found step-based intelligent tutoring systems nearly as effective as average human tutors, and both more effective than no tutoring but less effective than expert human tutors — related
• GURU, a tutor embodying expert-teacher behaviour, produced learning gains similar to small-group human teaching and better than classroom teaching alone — a narrower finding that bears on this claim
• A meta-analysis found that Intelligent Tutoring Systems outperformed, in aggregate, the other modes of instruction they were compared against, though publication bias may partly explain this — a broader claim this one bears on
• A meta-review found intelligent tutoring systems approaching the effectiveness of ordinary human tutors, whose effectiveness is below that of expert tutoring — related
• Non-expert human tutors follow a limited 5-step collaborative reasoning repertoire, with far more student question-asking than classroom instruction — related
• The introduction reports that the VCR tutor produced statistically significant performance gains in using the VCR, a rare result for ITSs at the time — a narrower finding that bears on this claim
• No significant difference in ICEMS performance scores between AI tutoring and expert instruction — a narrower finding that bears on this claim