Performance improvements from generative AI assistance diminished once the assistance was removed¶

Claim · All claims
Evidence · 1 study · theoretical r2 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? In a study of undergraduate peer feedback, performance improvements from generative AI-based assistance diminished once the assistance was removed. → Yan 2025

Evidence¶

Yan 2025¶

Yan, L., Greiff, S., Lodge, J. M., & Gašević, D. (2025). Distinguishing performance gains from learning when using generative AI. Nature Reviews Psychology, 4, 435–436. https://doi.org/10.1038/s44159-025-00467-5

q2 · i? · theoretical · r2

The article reports, citing Darvishi et al., a study finding significant boosts to undergraduate peer-feedback performance with generative AI assistance, which "diminished once the generative AI assistance was removed". No effect size is printed.

"Another study identified significant boosts to undergraduate student performance in providing quality peer feedback when using generative AI-based assistance[6]. However, these performance improvements diminished once the generative AI assistance was removed"

Discussion¶

Related Claims¶

• Students given AI assistance in peer feedback tended to rely on the AI-generated feedback and struggled to perform the task when the assistance was removed — possibly the same claim (merge candidate)
• Heavy reliance on generative AI can undermine learning: students with GPT-4 access performed better on practice problems but worse on exams once the tool was removed — a broader claim this one bears on
• AI-induced gains are not purely delegation: treated participants do not perform worse once AI is removed, and lower-education participants retain part of their gain, though a 0.200 SD gap re-emerges — reports the opposite
• Reviewed literature reports risks of overreliance, AI errors in complex tasks, and students' uncritical acceptance of AI-generated outputs — related
• The decisive empirical test of AI-augmented instruction frameworks is whether students retain representational competence when AI support is withdrawn — related
• AI assistance that substitutes for learning activities may improve immediate performance while impeding long-term skill development — a broader claim this one bears on
• Outsourcing productive struggle to AI risks cognitive debt, and reactive compliance-focused approaches are insufficient — a broader claim this one bears on
• High-school students given GPT-4 access solved more problems in the moment but performed worse than controls once the tool was removed (review attribution) — related
• Substituting tasks with generative AI may promote inclusion and free time for more demanding activities, but risks foundational-skill loss and presupposes prior knowledge — related
• Generative AI tools risk inducing overestimated abilities and diminishing training of foundational skills — related
• Rewards combined with feedback temporarily boosted reading performance, especially for struggling readers, with variable long-term effects — related