Practicing cover letter writing with an AI tool improved unassisted writing skill more than practicing without AI or not practicing, with benefits persisting one day later¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r3 · q4 · i1–i2
Currency · newest study 2025 · checked against its sources 2026-10-19 (automated, 2 of 2 entries) · not reviewed by a person

Subclaims¶

q4 i1 Participants who practiced with the AI tool wrote better cover letters on a no-AI test than those who practiced without AI (d=.38) or did not practice (d=.47). → Benjamin Lira 2025 q4 i2 The benefit persisted at a one-day follow-up test (d=.41 vs. no-AI practice; d=.46 vs. no practice). → Benjamin Lira 2025 (2)

Evidence¶

Benjamin Lira 2025¶

Benjamin Lira, Todd Rogers, Daniel G. Goldstein, Lyle Ungar, and Angela L. Duckworth. (2025). Coach not crutch: Evidence that AI can improve writing skill despite reducing effort. https://arxiv.org/abs/2502.02880 doi:10.48550/arxiv.2502.02880

q4 · i1 · causal · r3

Pre-registered randomized experiment (N=2,238, Prolific). After a lesson on five writing principles, participants were assigned to practice with AI, practice without AI, or no practice, then took a no-AI writing test. "wrote better cover letters" with d=.38 and d=.47.

"participants who had practiced with the AI tool wrote better cover letters than participants who either had practiced without the AI tool (d= .38,p<.001) or who had not practiced at all (d= .47,p<.001)"

Benjamin Lira 2025 (2)¶

Benjamin Lira, Todd Rogers, Daniel G. Goldstein, Lyle Ungar, and Angela L. Duckworth. (2025). Coach not crutch: Evidence that AI can improve writing skill despite reducing effort. https://arxiv.org/abs/2502.02880

q4 · i2 · causal · r3

One-day follow-up with 87% retention; attrition did not differ by condition (χ²=.68, p=.710). Participants "continued to outperform" those who practiced without AI (d=.41) and those who did not practice (d=.46). Condition × time interactions were non-significant (ps>.69).

"participants who had practiced with the AI tool the previous day continued to outperform those who had practiced without the tool (d= .41,p<.001) as well as those who had not practiced at all (d= .46,p< .001)"

Discussion¶

Learner Variables¶

• Writing Skills — outcome: instruction changes it

Related Claims¶

• Unrestricted GPT-4 access in high-school mathematics raised assisted practice performance but left students worse off after withdrawal — the general claim this finding bears on, with the studies behind it

• Immediate task success does not establish learning that persists without AI support — the general claim this finding bears on, with the studies behind it

• Practicing with AI improved unassisted writing skill more than practicing with Google Search for cover letter examples and tips — related
• Merely viewing an AI-generated writing example improved writing skill as much as actively practicing with the AI tool, with benefits persisting one day later — related
• AI-generated rewrites of cover letters were rated higher for writing quality and more likely to secure hypothetical interviews than rewrites by professional human editors — related
• Practicing with AI did not create an illusion of mastery: participants reported learning and skill levels similar to other conditions — related
• AI assistance reduces persistence: persistence costs concentrate among learners who used AI for direct solutions, not hints — related