Intelligent tutoring systems improve achievement over non-individualised instruction¶

Claim · All claims
Evidence · 2 studies · 2 quant-synthesis r2 · q4 · i2 medium
Currency · newest study 2016 · checked against its sources 2026-10-15 (automated, 2 of 2 entries) · not reviewed by a person

Learning environments that adjust task difficulty, sequencing, or support to individual learner performance can improve outcomes relative to fixed, one-size-fits-all sequences — but the effect depends heavily on how adaptation is implemented and for whom.

Subclaims¶

q4 i2 Across 107 effect sizes (14,321 participants), intelligent tutoring systems produced higher achievement than teacher-led large-group instruction (g = 0.42), non-ITS computer-based instruction (g = 0.57) and textbooks or workbooks (g = 0.35), but no advantage over individual human tutoring (g = -0.11) or small-group instruction (g = 0.05). → Ma et al. 2014

q4 i2 In 50 controlled evaluations the median effect of intelligent tutoring was 0.66 SD over conventional instruction, but the gain depended heavily on whether tests were locally developed or standardized, and was small where control treatments were nonconventional or implementations flawed. → Kulik & Fletcher 2016

Evidence¶

Ma et al. 2014¶

Ma, W., Adesope, O. O., Nesbit, J. C., & Liu, Q. (2014). Intelligent tutoring systems and learning outcomes: A meta-analysis. Journal of Educational Psychology, 106(4), 901–918. doi:10.1037/a0037123

q4 · meta-analysis · i2 · medium effect, g=0.42 vs teacher-led large-group instruction · n=107 effect sizes, 14,321 participants · quant-synthesis · r2

A meta-analysis of studies comparing students learning from intelligent tutoring systems (computer programs that model the learner's state to individualise instruction) with students in non-ITS learning environments. ITS were associated with greater achievement than teacher-led large-group instruction, other computer-based instruction and textbooks or workbooks, with positive effects at all education levels and in almost all subject domains. They did not outperform individualised human tutoring or small-group instruction, so the advantage is relative to non-individualised instruction rather than to adaptation delivered by people. Read from the abstract only.

Kulik & Fletcher 2016¶

Kulik, J. A., & Fletcher, J. D. (2016). Effectiveness of intelligent tutoring systems: A meta-analytic review. Review of Educational Research, 86(1), 42–78. doi:10.3102/0034654315581420

q4 · meta-analysis · i2 · medium effect, median 0.66 SD · n=50 controlled evaluations · quant-synthesis · r?

A meta-analysis of 50 controlled evaluations of intelligent computer tutoring systems against conventional instruction. The median effect raised test scores 0.66 standard deviations, from the 50th to the 75th percentile. The size of the gain depended strongly on whether outcomes were measured with locally developed or standardized tests, which the authors suggest makes test-instruction alignment a critical determinant of the effect; ten further evaluations with nonconventional control groups or flawed implementations showed small effects. Read from the abstract only.

Studies bearing on this claim¶

Does adaptive instruction raise achievement over non-individualised instruction or the same content in a fixed sequence? This table gathers the studies behind this claim and the 30 narrower claims that bear on it: 24 distinct studies (10 causal, 8 review, 4 quant-synthesis, 1 associational, 1 qualitative), q1–q4, 11 of 24 with an effect size coded. One row per study, codes as each entry records them; no effect is averaged and no verdict is drawn. Built by scripts/synthesize_claim.py from eval/synthesis/; a row that looks wrong is fixed on the claim it comes from.

Discussion¶

Mechanism. Adaptive systems plausibly improve outcomes through two routes: keeping learners in their zone of proximal development (tasks neither too easy nor too hard, consistent with cognitive load management), and ensuring mastery of prerequisites before advancing (as in adaptive mastery learning). Both routes predict the largest gains for learners who would otherwise be mismatched to a fixed sequence — struggling learners overwhelmed by uniform pacing, or advanced learners bored by it.

Boundary conditions. Adaptation is only as good as its model of the learner. Systems that adapt on shallow signals (response time, item counts) rather than diagnostic assessment of knowledge components may route learners poorly. There is also a plausible expertise-reversal concern: highly adaptive scaffolding that remains in place for already-proficient learners can become redundant and depress performance, mirroring the pattern documented for worked examples in expertise reversal effect. Adaptation should fade support as competence grows.

Open questions. Two meta-analyses of intelligent tutoring systems are recorded (Ma et al. 2014; Kulik & Fletcher 2016), both read from abstracts: tutoring systems outperformed large-group instruction, other computer-based instruction and textbooks, but not individual human tutoring or small-group instruction, and gains were smaller on standardized than on locally developed tests. No recorded study covers other adaptive systems, such as mastery-based platforms or adaptive quizzing. Key moderators still to establish include: which adaptation target (difficulty, pacing, feedback, content sequence) drives effects; whether gains persist beyond the adaptive period; and how outcomes compare across intelligent tutoring systems, mastery-based platforms, and simpler adaptive quizzing.

Related Claims¶

• Mastery learning effects are larger with a higher mastery standard, group-based pacing, local tests, and less feedback for the comparison group — the general claim this finding bears on, with the studies behind it

• Mastery learning improves achievement. — mastery-based adaptation is the classic mechanism by which adaptive sequencing helps

• Cognitive load theory — adaptation aims to keep load within learners' working-memory limits
• Expertise reversal effect — adaptive support can backfire when it persists for advanced learners
• Feedback improves learning — adaptive feedback delivery is a common implementation of adaptation
• Adaptive learning — the design principle this claim evaluates empirically
• Adaptive difficulty — difficulty adjustment is the most common adaptation target in practice
• Contingent scaffolding improves learning more than fixed or absent support. — related
• Human tutoring has a medium effect over no tutoring, and in one small study students learned as well when tutors only prompted them as when tutors also explained and gave feedback — related
• The survey reports, citing Lee and Brunskill, that individualized BKT in an intelligent tutoring system reduced by about half the questions required for 20% of students to achieve mastery. — related
• Peer Tutoring Improves Achievement — related
• Scaffolding improves learning — related
• Early objective and subjective feedback helps evaluate program effectiveness and prevents small problems from growing — related
• The survey reports, citing Long and Aleven, that students who used DragonBox enjoyed the experience more, while students who used the Lynnette intelligent tutoring system performed significantly better on the test. — related
• Mastery Learning Improves Outcomes — related
• The measured advantage of computer-based instruction over conventional teaching shrinks when the same teacher teaches both versions — related
• System compensation as implemented is not a satisfactory adaptive variable — related
• Strategic speeded practice outperformed strategic non-speeded practice within Galaxy Math, with a +0.57 effect size for the disseminated speeded version — related
• Platform-enabled experimentation research draws on multiple intellectual lineages, including intelligent tutoring systems, formative feedback, and exemplar platforms like ASSISTments — related
• An intelligent reading tutor used 20 minutes a day offered time efficiencies over conventional human tutoring of 30 or more minutes a day — related
• Adaptive scaffolding policies (BKT and DRL) significantly improve posttest performance over a non-adaptive control in a logic ITS — a narrower finding that bears on this claim
• AI-supported adaptive systems are reported to enhance adult learner engagement, motivation, and outcomes when aligned with learner goals and prior knowledge — a broader claim this one bears on
• AURA's within-session reinforcement learning improved composite response quality over non-adaptive baselines (p = 0.044, d = 0.66), with fewer specification prompts and more validation behavior — a narrower finding that bears on this claim
• Intelligent tutoring systems are reported to improve learning outcomes, especially in structured knowledge domains — a broader claim this one bears on
• Intelligent tutoring systems can improve learning outcomes, particularly with immediate actionable feedback — a broader claim this one bears on
• In university physics education, AI has been applied for tutoring and explanations, formative feedback and scaffolding, collaborative problem solving, simulation and modeling, instructional design, and AI literacy development — related
• One-to-one human tutoring lifts an ordinary student well beyond the average classroom with an effect of about d = 0.79 (VanLehn, 2011, as reported) — related
• The system has no empirical evaluation yet: profile accuracy has not been assessed and learning outcomes remain an open question — related
• Meta-analytic evidence indicates intelligent tutoring systems yield positive learning outcomes compared with non-ITS conditions — a broader claim this one bears on
• Adaptive learning functions as the algorithmic engine that operationalizes personalized learning — related
• Intelligent tutoring systems are a mature, technologically sophisticated implementation of AI-driven personalized learning with mostly positive outcomes in programming, mathematics, and STEM — a broader claim this one bears on
• Anderson's structured cognitive tutoring systems and Papert's constructionist approach embody contrasting positions on control and learner agency — related
• Integrating cognitive scores with non-cognitive learner attributes enables the system to dynamically adapt AI-generated explanations in tone, depth and instructional strategy to individual learner performance — a narrower finding that bears on this claim
• Using the ITS reduced executive state-math anxiety and improved algebra learning across conditions — a narrower finding that bears on this claim
• Learner engagement was strengthened when the AI tutor adapted to learners' pace and prior knowledge — related
• One pilot instructor reported the adaptive proctor differentiated students at multiple proficiency boundaries and decoupled conceptual mastery from computational speed — a narrower finding that bears on this claim
• Within sociocultural theory, teacher feedback and AI-assisted peer feedback function as heterogeneous scaffolds with distinct natures and complementary advantages — related
• AutoTutor achieved learning gains comparable to human tutors in physics and computer literacy — related
• A narrative review of intelligent tutoring systems asserts, citing one source, that students in personalized learning environments outperform peers in standardized settings — related
• In one quasi-experimental VR shooting study (n = 50), reported in a review, rule-based dynamic difficulty outperformed progressive difficulty — a narrower finding that bears on this claim
• A technical review reports that one reinforcement-learning adaptive platform raised average course grades by 12%, task completion by 20% and cut dropout by 10% against static curricula (uncontrolled case report) — a narrower finding that bears on this claim
• ITS significantly outperform large-group human instruction, individual computer-based instruction, and textbooks or workbooks, but not small-group or individual human tutoring — possibly the same claim (merge candidate)
• Learning with intelligent tutoring systems yields a moderate, statistically significant overall advantage in achievement over non-ITS modes of instruction — possibly the same claim (merge candidate)
• Intelligent tutoring raised test scores by a median of 0.66 standard deviations over conventional instruction across 50 controlled evaluations — related
• Adaptive fading of worked examples based on current skill level produced better learning outcomes than fixed fading or tutored problem solving — related
• AI-enabled personalized STEM education yields a statistically significant overall effect of g = 0.455, which its authors class as small, compared with traditional non-AI instruction in K-12 schools — related
• Intelligent tutoring systems have been shown to improve upon typical classroom instruction — possibly the same claim (merge candidate)
• Intelligent tutoring systems show moderate learning benefits but are less effective than human tutoring — possibly the same claim (merge candidate)
• An adaptive AI-based STEM instructional program produced statistically significant differences favoring the experimental group across all deep learning dimensions in sixth-grade science — a narrower finding that bears on this claim
• Adaptive versus non-adaptive ITS condition showed no significant differences in mental effort, subjective difficulty, or enjoyment — related
• Meta-analytic evidence reviewed reports medium-to-large positive effects of AI-enabled adaptive learning on cognitive outcomes (g = 0.50–0.70) — related
• Introducing human-AI tutoring improves relative MAP academic growth by 61% over an AI-only baseline within the IK bandwidth — related
• Six AI tutors individually matched human tutoring, with Gemma 4 31B equivalent at 918× lower cost per percentage point of learning gain — a narrower finding that bears on this claim
• Computer-based scaffolding shows a consistently positive effect on cognitive outcomes in STEM education — related
• Automated real-time feedback in an adaptive diagnostic ITS produced greater progression on a SOLO-based mathematics assessment than the same system without real-time feedback — a narrower finding that bears on this claim
• ITS groups outperformed conventional-instruction controls in 92% of the 50 evaluations, with gains above 0.25 SD in 78% — related
• In a Moore, Oklahoma experiment, Cognitive Tutor students scored higher on the ETS Algebra I End-of-Course Assessment, received higher grades, reported greater confidence, and showed effect sizes of approximately 0.4 SDs — a narrower finding that bears on this claim
• Multi-agent tutoring architectures show superior behaviour modelling and feedback compared to single-agent or rule-based tutors — a narrower finding that bears on this claim
• In a randomized experiment, personalized paths reduced learning time by 22.0% and improved post-test scores versus a fixed sequence — a narrower finding that bears on this claim
• The DARPA Digital Tutor, an ITS for U.S. Navy information systems technicians, produced the largest effects in the data set (average ES of 1.97 and 3.18 in two summative evaluations) — a narrower finding that bears on this claim
• Learning benefits of Cognitive Tutor interactivity are supported to varying extents by a growing body of experimental studies — a narrower finding that bears on this claim
• Intelligent tutoring systems with dynamic student modeling increase learning gains over regular instruction — possibly the same claim (merge candidate)
• ITS augmentation: meta-analyses find medium to large learning benefits of ITS over unsupported learning and small to medium gains over less adaptive computer-assisted learning — related
• ITS substitution: meta-analyses find no significant cognitive learning differences between ITS and human tutoring or small-group instruction — related
• Cognitive-load-based quiz adaptation raised learning gains while domain-specific confidence-based adaptation did not — a narrower finding that bears on this claim