Students use standard GPT-4 access as a crutch—copying answers rather than engaging with the material—which explains the harm to learning¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r3 · q2–q3
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? GPT Base's logical errors hurt practice performance but did not spill over to corresponding exam problems, and arithmetic and logical errors had similar practice effects—patterns the authors read as answer copying. → Bastani Hamsa 2025 q3 i? Message analysis shows GPT Base students most often simply ask for the answer, while GPT Tutor students increasingly ask for help and attempt answers independently. → Bastani Hamsa 2025 (2)

Evidence¶

Bastani Hamsa 2025¶

Bastani Hamsa, Bastani Osbert, Sungu Alp, Ge Haosen, Kabakcı Özge, Mariman Rei. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences of the United States of America. https://doi.org/10.1073/pnas.2422633122

q3 · i? · causal · r3

Problem-level regression with GPT Base error-rate interaction terms (13,484 practice / 11,392 exam observations). The "GPT Base × Logical Error Rate" interaction was −0.448 (P < 0.01) for practice but not significant for exams, supporting the crutch interpretation.

"while GPT Base’s logical errors affect performance on the practice problems, we find no evidence that this effect spills over to the corresponding exam problems"

Bastani Hamsa 2025 (2)¶

Bastani Hamsa, Bastani Osbert, Sungu Alp, Ge Haosen, Kabakcı Özge, Mariman Rei. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences of the United States of America. https://doi.org/10.1073/pnas.2422633122

q2 · i? · causal · r3

NLP clustering of student messages with the tutors, shown in Fig. 3 and SI Appendix Fig. C.4; GPT Base conversations were largely superficial (asking for answers) while GPT Tutor conversations were substantially more often nonsuperficial.

"students in GPT Base most often simply ask for the answer; in contrast, students in GPT Tutor learn to interact more substantively with the tutoring tool over time by asking for help and independently attempting to solve the problem."

Discussion¶

Related Claims¶

• Students do not perceive the learning harm from copying GPT solutions and overestimate their exam performance — related
• Persistence and performance costs concentrate among participants who used the AI to obtain direct solutions, while hint-users resemble controls — related
• How a user engages with AI matters more than whether: structured, deliberate engagement can mitigate or reverse offloading effects — related
• Unguarded answer-giving AI harmed unaided exam performance while a guarded version of the same model erased the harm (Bastani et al., 2025, as reported) — related
• Greater answer offloading during learning is associated with lower subsequent unaided performance (exploratory) — related
• Unfettered GPT-4 access during practice harms subsequent unassisted exam performance in high school mathematics — related
• Among GPT-assisted students, proactive and critical engagement patterns are associated with higher exam scores while limited engagement shows little benefit relative to control — related
• Access to a standard GPT-4 chat tutor during practice substantially improves practice performance in high school mathematics — related
• A guardrailed GPT-4 tutor with teacher-designed hints improves assisted practice performance more than a standard GPT-4 interface — related
• Chatbot access during mathematics practice improved practice performance but lowered subsequent unaided exam performance, and a hint-giving interface eliminated the penalty — related