Early AutoTutor versions gained about 0.5 σ over textbook controls, with gains depending on test type (cloze 0.64 σ, deep 0.28 σ, shallow 0.15 σ)¶

Claim · All claims
Evidence · 1 study (2 entries) · review r2 · q2 · i2 medium
Currency · newest study 2014 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i2 AutoTutor 1.1 and 2.0 both showed learning gains of approximately 0.5 σ over controls who studied relevant textbook chapters for equivalent time. → Nye 2014 q2 i2 Gains depended on evaluation test type: 0.15 σ for shallow questions, 0.28 σ for deep questions, and 0.64 σ for cloze questions. → Nye 2014 (2)

Evidence¶

Nye 2014¶

Nye, B. D., Graesser, A. C., & Hu, X. (2014). AutoTutor and Family: A Review of 17 Years of Natural Language Tutoring. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-014-0029-5

q2 · i2 · review · r2

Review reports evaluations of AutoTutor 1.1 and 2.0 in computer literacy against textbook-studying controls; the two versions performed comparably. Printed effect size: approximately 0.5 σ.

"Despite these differences, both versions performed comparably to each other and both showed learning gains of approximately 0.5 σ over controls who studied relevant textbook chapters for an equivalent amount of time"

Nye 2014 (2)¶

Nye, B. D., Graesser, A. C., & Hu, X. (2014). AutoTutor and Family: A Review of 17 Years of Natural Language Tutoring. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-014-0029-5

q2 · i2 · review · r2

Review reports the test-type dependence of the early computer-literacy evaluations' effect sizes, foreshadowing higher gains for deep reasoning. Printed effect sizes: 0.15 σ, 0.28 σ, 0.64 σ.

"These effect sizes were highly dependent on the type of evaluation test question, with gains of 0.15 σ for shallow questions, 0.28 σ for deep questions, and 0.64 σ for cloze questions"

Discussion¶

Related Claims¶

• Affective support in AutoTutor benefits low-knowledge learners but not high-knowledge learners, and gaze-based disengagement reactivity yields significant gains only on deep learning — related
• AutoTutor produces learning gains averaging about 0.8 standard deviations over static-textbook controls, and bystanders cannot distinguish its tutoring turns from a human tutor's — related
• AutoTutor yields average learning gains of 0.3 to 0.8 sigma compared to reading text for an equivalent amount of time — a broader claim this one bears on
• In WHY2 physics studies, AutoTutor outperformed read-textbook and no-materials controls and performed comparably to WHY2/Atlas and non-expert human tutors — related
• The review reports that AutoTutor improves learning by nearly one letter grade compared with reading a textbook for an equivalent time or with a pretest — related