In WHY2 physics studies, AutoTutor outperformed read-textbook and no-materials controls and performed comparably to WHY2/Atlas and non-expert human tutors¶

Claim · All claims
Evidence · 1 study (2 entries) · review r2 · q2 · i2 medium
Currency · newest study 2014 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i2 A pretest/posttest study found learning gains of 0.61 σ over the no-materials control posttest mean and 1.22 σ over the read-textbook posttest mean for WHY2/AutoTutor. → Nye 2014 q2 i? WHY2/AutoTutor and WHY2/Atlas, despite significantly different dialog management and semantic matching approaches, performed comparably to each other and to non-expert human tutors. → Nye 2014 (2)

Evidence¶

Nye 2014¶

Nye, B. D., Graesser, A. C., & Hu, X. (2014). AutoTutor and Family: A Review of 17 Years of Natural Language Tutoring. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-014-0029-5

q2 · i2 · review · r2

Review reports the WHY2/AutoTutor pretest/posttest study against read-textbook and no-physics-materials controls, with normalized learning gains of 0.32 σ for AutoTutor versus 0.04 σ for read-textbook. Printed effect sizes: 0.61 σ and 1.22 σ.

"This study found learning gains of 0.61 σ for students in the AutoTutor condition over the control posttest mean and 1.22 σ over the read-textbook posttest mean"

Nye 2014 (2)¶

Nye, B. D., Graesser, A. C., & Hu, X. (2014). AutoTutor and Family: A Review of 17 Years of Natural Language Tutoring. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-014-0029-5

q2 · i? · review · r2

Review reports that WHY2/AutoTutor's LSA-based statistical matching and WHY2/Atlas's finite-state, physics-specific hybrid classifier yielded comparable performance; no effect size printed for this comparability.

"Despite using significantly different approaches to dialog management and semantic matching, both systems performed comparably"

Discussion¶

Related Claims¶

• AutoTutor produces learning gains averaging about 0.8 standard deviations over static-textbook controls, and bystanders cannot distinguish its tutoring turns from a human tutor's — related
• Early AutoTutor versions gained about 0.5 σ over textbook controls, with gains depending on test type (cloze 0.64 σ, deep 0.28 σ, shallow 0.15 σ) — related
• AutoTutor's learning gains are virtually equivalent to those of 1-to-1 human tutoring in direct comparisons — related
• The review reports that AutoTutor improves learning by nearly one letter grade compared with reading a textbook for an equivalent time or with a pretest — a broader claim this one bears on
• AutoTutor achieved learning gains comparable to human tutors in physics and computer literacy — a broader claim this one bears on
• Meta-analytic evidence indicates intelligent tutoring systems yield positive learning outcomes compared with non-ITS conditions — a broader claim this one bears on
• AutoTutor yields average learning gains of 0.3 to 0.8 sigma compared to reading text for an equivalent amount of time — a broader claim this one bears on
• ITS significantly outperform large-group human instruction, individual computer-based instruction, and textbooks or workbooks, but not small-group or individual human tutoring — a broader claim this one bears on
• Intelligent tutoring systems improve achievement over non-individualised instruction — a broader claim this one bears on
• Content Acquisition Podcasts produced significantly higher posttest scores than textbook-based instruction for preservice teachers — related