Immediate task success does not establish learning that persists without AI support¶

Claim · All claims
Evidence · 1 study · review r2 · q1
Currency · newest study 2026 · checked against its sources 2026-10-17 (automated, 1 of 1 entry) · not reviewed by a person

Subclaims¶

q1 i? Across the focused evidence set, even stronger behavioral designs generally stop at immediate accuracy rather than delayed, unaided learning outcomes such as retention, transfer, or independent error detection. → Wei J and Shang Y 2026

Evidence¶

Wei J and Shang Y 2026¶

Wei J and Shang Y (2026). Beyond checking: verification quality, reliance calibration, and learning in generative AI-assisted higher education. Front. Psychol. 17:1965371. https://doi.org/10.3389/fpsyg.2026.1965371

q1 · i? · review · r2

The review's analysis of its Table 1 studies: several observe an immediate answer, written product, acceptance decision, or quiz score, but such outcomes "do not establish retention or transfer". Independent learning is treated as an umbrella domain including retention, transfer, unaided performance, and independent error detection after AI support is withdrawn.

"Across Table 1, even stronger behavioral designs generally stop at immediate accuracy rather than delayed, unaided learning."

Studies bearing on this claim¶

Does an advantage measured at the end of instruction still hold on a delayed, unaided test? This table gathers the studies behind this claim and the 8 narrower claims that bear on it: 9 distinct studies (7 causal, 2 review), q1–q4, 4 of 9 with an effect size coded. One row per study, codes as each entry records them; no effect is averaged and no verdict is drawn. Built by scripts/synthesize_claim.py from eval/synthesis/; a row that looks wrong is fixed on the claim it comes from.

Discussion¶

Related Claims¶

• AI access raises immediate unaided test scores by 7.2 percentage points (0.28 SD) after a timed learning phase — related
• Unguarded answer-giving AI harmed unaided exam performance while a guarded version of the same model erased the harm (Bastani et al., 2025, as reported) — related
• AI assistance that substitutes for learning activities may improve immediate performance while impeding long-term skill development — a broader claim this one bears on
• Among GPT-assisted students, proactive and critical engagement patterns are associated with higher exam scores while limited engagement shows little benefit relative to control — related
• Gains from an intensive period of second-language study persist for weeks without rehearsal. — related
• AI-induced gains are not purely delegation: treated participants do not perform worse once AI is removed, and lower-education participants retain part of their gain, though a 0.200 SD gap re-emerges — related
• One week later, the AI-access group's lead on unaided assessments shrinks to 3.9 pp (0.21 SD), short of statistical significance (p = 0.068) — related
• Agentivism generates six empirically testable propositions linking delegation structure, verification, reconstruction, and reduced-support transfer to learning outcomes — a narrower finding that bears on this claim
• Proposition 1: outcome-aligned ownership performances will predict later individual transfer beyond AI-assisted product quality alone — a narrower finding that bears on this claim
• Developers who used AI assistance scored 17% lower on a subsequent comprehension assessment than developers who completed the same tasks without AI, per the cited controlled study — a narrower finding that bears on this claim
• AI-assisted performance during skill-building does not transfer to subsequent unassisted performance — a narrower finding that bears on this claim
• Practicing cover letter writing with an AI tool improved unassisted writing skill more than practicing without AI or not practicing, with benefits persisting one day later — a narrower finding that bears on this claim
• Effective AI use positively influences sustainable learning performance — related
• The immediate advantage of deep-processing prompts does not persist at a 5-month follow-up, with a Bayesian test confirming the null — a narrower finding that bears on this claim
• The retention gap persisted at 48 hours with no differential forgetting between groups (39% vs 52%) — a narrower finding that bears on this claim
• AI-generated and prebuilt simulations produced higher immediate conceptual-assessment scores than physical laboratory work, but the advantage did not persist in later examinations — related
• Learning should be assessed with retention and transfer tests rather than current practice performance — related
• Merely viewing an AI-generated writing example improved writing skill as much as actively practicing with the AI tool, with benefits persisting one day later — related
• Participating in a relevant group discussion, whether explaining or listening, raises immediate recall relative to an irrelevant discussion, but this advantage does not persist one month later — a narrower finding that bears on this claim
• Evidence on GenAI in chemistry education is concentrated in short-term outcomes, with durable transfer and retention rarely tested — possibly the same claim (merge candidate)
• Intensive AI assistance predicts strong task performance even with low engagement, but follow-up performance is substantially higher only when intensive AI use is combined with sustained task engagement — a narrower finding that bears on this claim