Merely viewing an AI-generated writing example improved writing skill as much as actively practicing with the AI tool, with benefits persisting one day later¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r3 · q4 · i0 negligible
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q4 i0 Participants who viewed an AI-generated example without editing it improved as much as participants who practiced with the AI tool (d=.03, p=.883). → Benjamin Lira 2025 q4 i0 At one-day follow-up, the example-only and AI-practice conditions did not differ from each other (d=.02, p=.830). → Benjamin Lira 2025 (2)

Evidence¶

Benjamin Lira 2025¶

Benjamin Lira, Todd Rogers, Daniel G. Goldstein, Lyle Ungar, and Angela L. Duckworth. (2025). Coach not crutch: Evidence that AI can improve writing skill despite reducing effort. https://arxiv.org/abs/2502.02880 doi:10.48550/arxiv.2502.02880

q4 · i0 · causal · r3

Pre-registered replication and extension (N=2,003, Prolific) replacing no-practice with an example-only condition. Participants who merely saw an example "improved as much as participants who had practiced with the AI tool" (d=.03, p=.883).

"they improved as much as participants who had practiced with the AI tool (and could edit its output,d= .03,p= .883)"

Benjamin Lira 2025 (2)¶

Benjamin Lira, Todd Rogers, Daniel G. Goldstein, Lyle Ungar, and Angela L. Duckworth. (2025). Coach not crutch: Evidence that AI can improve writing skill despite reducing effort. https://arxiv.org/abs/2502.02880

q4 · i0 · causal · r3

One-day follow-up subsample (n=633 of n=800 recontacted). Both AI conditions "continued to outperform" practice without AI, and the example-only and AI-practice conditions did not differ from each other (d=.02, p=.830). Condition × time interactions were non-significant (ps>.96).

"Participants who had practiced with the AI tool (d= .29,p= .006) and participants who had simply seen an AI example (d= .32,p= .003), both continued to outperform those who had practiced without the tool"

Discussion¶

Learner Variables¶

• Writing Skills — outcome: instruction changes it

Related Claims¶

• Example-based sequences outperform problem-only practice for novices, and fading support is argued to aid transfer — the general claim this finding bears on, with the studies behind it
• Practicing cover letter writing with an AI tool improved unassisted writing skill more than practicing without AI or not practicing, with benefits persisting one day later — related
• Practicing with AI did not create an illusion of mastery: participants reported learning and skill levels similar to other conditions — related
• Active participation in interaction matters for second language developmental outcomes — related
• Practicing with AI improved unassisted writing skill more than practicing with Google Search for cover letter examples and tips — related
• Immediate task success does not establish learning that persists without AI support — related