The DARPA Digital Tutor, an ITS for U.S. Navy information systems technicians, produced the largest effects in the data set (average ES of 1.97 and 3.18 in two summative evaluations)¶

Claim · All claims
Evidence · 1 study · quant-synthesis r3 · q3 · i3 large
Currency · newest study 2016 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i3 In two summative evaluations of the DARPA Digital Tutor against standard Navy classroom courses, average effect sizes were ES = 1.97 and ES = 3.18 standard deviations. → Kulik 2016

Evidence¶

Kulik 2016¶

Kulik, J. A., & Fletcher, J. D. (2016). Effectiveness of Intelligent Tutoring Systems: A Meta-Analytic Review. Review of Educational Research. https://doi.org/10.3102/0034654315581420

q3 · i3 · quant-synthesis · r3

Two summative field evaluations compared end-of-course test scores from Digital Tutor courses (8 and 16 weeks) with longer standard classroom courses, measured on locally developed third-party written and oral tests. The printed effect sizes were ES = 1.97 and ES = 3.18: "Average ES in the first evaluation was 1.97"; both were outliers in the data set.

"Average ES in the first evaluation was 1.97 (Fletcher, 2011); average ES in the second evaluation was 3.18 (Fletcher & Morrison, 2012)."

Discussion¶

Related Claims¶

• ITS evaluation results depend strongly on test type: effects are large on locally developed tests (ES = 0.73) and small on standardized tests (ES = 0.13) — related
• Intelligent tutoring raised test scores by a median of 0.66 standard deviations over conventional instruction across 50 controlled evaluations — a broader claim this one bears on
• ITS groups outperformed conventional-instruction controls in 92% of the 50 evaluations, with gains above 0.25 SD in 78% — related
• Intelligent tutoring systems improve achievement over non-individualised instruction — a broader claim this one bears on