Practicing with AI did not create an illusion of mastery: participants reported learning and skill levels similar to other conditions¶
Claim · All claims
Evidence · 1 study · causalr3·q4·i0negligible
Currency · newest study 2025 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q4 i0 Participants who practiced with AI reported learning about as much as those who practiced without it or not practiced, and judged their writing skill similarly (|ds|≤0.10, ps>.05). → Benjamin Lira 2025
Evidence¶
Benjamin Lira 2025¶
Benjamin Lira, Todd Rogers, Daniel G. Goldstein, Lyle Ungar, and Angela L. Duckworth. (2025). Coach not crutch: Evidence that AI can improve writing skill despite reducing effort. https://arxiv.org/abs/2502.02880 doi:10.48550/arxiv.2502.02880
q4 · i0 · causal · r3
Exit questionnaire in the Study 2 experiment (N=2,238). Participants who practiced with AI "reported learning about as much as participants who had practiced without it or not practiced", with all self-report differences |ds|≤0.10 and non-significant.
"participants who had practiced with AI reported learning about as much as participants who had practiced without it or not practiced. Likewise, they judged their writing skill similarly to those who had either practiced alone or done no practice at all (|ds|≤ 0.10,ps >.05)"
Discussion¶
Related Claims¶
- Access to AI advice roughly doubled reported confidence in answers in one experiment — the general claim this finding bears on, with the studies behind it
- Merely viewing an AI-generated writing example improved writing skill as much as actively practicing with the AI tool, with benefits persisting one day later — related
- Fluent Illusions Mislead Self Assessment — related
- Practicing cover letter writing with an AI tool improved unassisted writing skill more than practicing without AI or not practicing, with benefits persisting one day later — related
- People overestimate how much time AI assistance saves (speedup illusion), driven by miscalibration about AI-assisted completion time while independent completion time is well calibrated — related
- No significant difference in perceived learning across the three modes (p = .845), despite the unrestricted mode's nearly double objective learning delta — related
- Students' perceptions of mastery, emotions, and satisfaction were similarly high across all three feedback conditions — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
no-illusion-of-mastery-from-ai-practice |
title |
Practicing with AI did not create an illusion of mastery: participants reported learning and skill levels similar to other conditions |
description |
Practicing with AI did not create an illusion of mastery: participants reported learning and skill levels similar to other conditions |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2025 |
source_checked |
2026-10-20 |
generated |
by process:wiki-ingest on 2026-10-10 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — benjamin-lira-2025 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.