Access to a standard GPT-4 chat tutor during practice substantially improves practice performance in high school mathematics¶
Claim · All claims
Evidence · 1 study · causalr3·q3
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q3 i? Students with GPT Base access scored 0.137 higher (out of 1) on assisted practice problems than control students, a 48% improvement over the control mean. → Bastani Hamsa 2025
Evidence¶
Bastani Hamsa 2025¶
Bastani Hamsa, Bastani Osbert, Sungu Alp, Ge Haosen, Kabakcı Özge, Mariman Rei. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences of the United States of America. https://doi.org/10.1073/pnas.2422633122
q3 · i? · causal · r3
Intention-to-treat regression from the RCT (2,848 observations, classroom-clustered SEs) on assisted practice performance; the GPT Base coefficient was 0.137 (P < 0.01) against a "mean performance of 0.28". No standardized effect size is printed.
"GPT Base and GPT Tutor substantially increased student scores in the GPT-assisted practice sessions by 0.137 and 0.361 (out of 1), respectively, relative to the control arm that only had access to textbooks (mean performance of 0.28)."
Discussion¶
Related Claims¶
- A guardrailed GPT-4 tutor with teacher-designed hints improves assisted practice performance more than a standard GPT-4 interface — related
- Unrestricted GPT-4 access in high-school mathematics raised assisted practice performance but left students worse off after withdrawal — related
- Unfettered GPT-4 access during practice harms subsequent unassisted exam performance in high school mathematics — related
- Observable output alone is insufficient evidence of learning when generative AI can produce sophisticated work without underlying cognitive change — related
- Guardrails in the GPT Tutor largely eliminate the negative exam effect, though no positive exam effect is observed — related
- High-school students given GPT-4 access solved more problems in the moment but performed worse than controls once the tool was removed (review attribution) — a broader claim this one bears on
- Students use standard GPT-4 access as a crutch—copying answers rather than engaging with the material—which explains the harm to learning — related
- Students do not perceive the learning harm from copying GPT solutions and overestimate their exam performance — related
- Chatbot access during mathematics practice improved practice performance but lowered subsequent unaided exam performance, and a hint-giving interface eliminated the penalty — a broader claim this one bears on
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
gpt-base-improves-assisted-practice-performance |
title |
Access to a standard GPT-4 chat tutor during practice substantially improves practice performance in high school mathematics |
description |
Access to a standard GPT-4 chat tutor during practice substantially improves practice performance in high school mathematics |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2025 |
generated |
by process:wiki-ingest on 2026-10-12 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — bastani-hamsa-2025 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.