Students do not perceive the learning harm from copying GPT solutions and overestimate their exam performance¶

Claim · All claims
Evidence · 1 study · causal r3 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? GPT Base students who performed worse on the exam did not perceive worse performance or learning, and GPT Tutor students perceived improvements that did not occur. → Bastani Hamsa 2025

Evidence¶

Bastani Hamsa 2025¶

Bastani Hamsa, Bastani Osbert, Sungu Alp, Ge Haosen, Kabakcı Özge, Mariman Rei. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences of the United States of America. https://doi.org/10.1073/pnas.2422633122

q2 · i? · causal · r3

End-of-session student surveys reported in SI Appendix Appendix B.3; the article notes "This mismatch between perceived and actual learning has been observed in other settings." Details of the survey analysis are in supplementary material not fully printed here.

"While students in the GPT Base arm performed worse on the exam (relative to the control arm), they did not perceive that they performed worse or learned less."

Discussion¶

Related Claims¶

• Access to AI advice roughly doubled reported confidence in answers in one experiment — the general claim this finding bears on, with the studies behind it
• Students use standard GPT-4 access as a crutch—copying answers rather than engaging with the material—which explains the harm to learning — related
• Review reports a field experiment in which GPT access improved supported practice but was followed by poorer unaided test performance, mitigated by a guarded tutor — related
• Guardrails in the GPT Tutor largely eliminate the negative exam effect, though no positive exam effect is observed — related
• Unfettered GPT-4 access during practice harms subsequent unassisted exam performance in high school mathematics — related
• A guardrailed GPT-4 tutor with teacher-designed hints improves assisted practice performance more than a standard GPT-4 interface — related
• Greater answer offloading during learning is associated with lower subsequent unaided performance (exploratory) — related
• Access to a standard GPT-4 chat tutor during practice substantially improves practice performance in high school mathematics — related
• Among GPT-assisted students, proactive and critical engagement patterns are associated with higher exam scores while limited engagement shows little benefit relative to control — related
• High-school students given GPT-4 access solved more problems in the moment but performed worse than controls once the tool was removed (review attribution) — related
• Chatbot access during mathematics practice improved practice performance but lowered subsequent unaided exam performance, and a hint-giving interface eliminated the penalty — related