Unfettered GPT-4 access during practice harms subsequent unassisted exam performance in high school mathematics¶
Claim · All claims
Evidence · 1 study · causalr3·q3
Currency · newest study 2025 · checked against its sources 2026-10-18 (automated, 1 of 1 entry) · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q3 i? Students in the GPT Base arm scored 0.054 lower (out of 1) on the unassisted exam than control students, a 17% reduction. → Bastani Hamsa 2025
Evidence¶
Bastani Hamsa 2025¶
Bastani Hamsa, Bastani Osbert, Sungu Alp, Ge Haosen, Kabakcı Özge, Mariman Rei. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences of the United States of America. https://doi.org/10.1073/pnas.2422633122
q3 · i? · causal · r3
Preregistered primary analysis: intention-to-treat regression on unassisted exam performance (2,848 observations, classroom-clustered SEs); GPT Base coefficient −0.054 (P < 0.05) against a control mean of 0.321. No standardized effect size is printed.
"student performance in the GPT Base arm degraded by 0.054 (out of 1) relative to that of the control arm. In other words, GPT Base diminished the average control student’s performance on the unassisted exam by 17%."
Discussion¶
Related Claims¶
- Access to a standard GPT-4 chat tutor during practice substantially improves practice performance in high school mathematics — related
- A guardrailed GPT-4 tutor with teacher-designed hints improves assisted practice performance more than a standard GPT-4 interface — related
- Guardrails in the GPT Tutor largely eliminate the negative exam effect, though no positive exam effect is observed — related
- Reviews citing one randomised trial (Bastani et al. 2025) report that unrestricted GPT-4 access in high-school mathematics raised assisted practice performance but left students worse off after withdrawal — possibly the same claim (merge candidate)
- Students do not perceive the learning harm from copying GPT solutions and overestimate their exam performance — related
- Students use standard GPT-4 access as a crutch—copying answers rather than engaging with the material—which explains the harm to learning — related
- Greater answer offloading during learning is associated with lower subsequent unaided performance (exploratory) — related
- Developers who used AI assistance scored 17% lower on a subsequent comprehension assessment than developers who completed the same tasks without AI, per the cited controlled study — related
- Chatbot access during mathematics practice improved practice performance but lowered subsequent unaided exam performance, and a hint-giving interface eliminated the penalty — a broader claim this one bears on
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
gpt-base-harms-unassisted-exam-performance |
title |
Unfettered GPT-4 access during practice harms subsequent unassisted exam performance in high school mathematics |
description |
Unfettered GPT-4 access during practice harms subsequent unassisted exam performance in high school mathematics |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2025 |
source_checked |
2026-10-18 |
generated |
by process:wiki-ingest on 2026-10-12 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — bastani-hamsa-2025 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.