Guardrails in the GPT Tutor largely eliminate the negative exam effect, though no positive exam effect is observed¶
Claim · All claims
Evidence · 1 study · causalr3·q3
Currency · newest study 2025 · checked against its sources 2026-10-18 (automated, 1 of 1 entry) · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q3 i? GPT Tutor students' unassisted exam performance was statistically indistinguishable from control (point estimate −0.004), so the guardrails mitigated but did not reverse the harm into a benefit. → Bastani Hamsa 2025
Evidence¶
Bastani Hamsa 2025¶
Bastani Hamsa, Bastani Osbert, Sungu Alp, Ge Haosen, Kabakcı Özge, Mariman Rei. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences of the United States of America. https://doi.org/10.1073/pnas.2422633122
q3 · i? · causal · r3
Intention-to-treat regression on unassisted exam performance; the GPT Tutor coefficient was −0.004 and not statistically significant. The article states "this negative effect is essentially eradicated in the GPT Tutor arm, though we still do not observe a positive effect."
"Student performance in the GPT Tutor arm was statistically indistinguishable from that of the control arm, and the point estimate was smaller by an order of magnitude (−0.004)."
Discussion¶
Related Claims¶
- A guardrailed GPT-4 tutor with teacher-designed hints improves assisted practice performance more than a standard GPT-4 interface — related
- Unfettered GPT-4 access during practice harms subsequent unassisted exam performance in high school mathematics — related
- Access to a standard GPT-4 chat tutor during practice substantially improves practice performance in high school mathematics — related
- Reviews citing one randomised trial (Bastani et al. 2025) report that unrestricted GPT-4 access in high-school mathematics raised assisted practice performance but left students worse off after withdrawal — related
- Students do not perceive the learning harm from copying GPT solutions and overestimate their exam performance — related
- Among GPT-assisted students, proactive and critical engagement patterns are associated with higher exam scores while limited engagement shows little benefit relative to control — related
- Chatbot access during mathematics practice improved practice performance but lowered subsequent unaided exam performance, and a hint-giving interface eliminated the penalty — a broader claim this one bears on
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
gpt-tutor-exam-effect-null |
title |
Guardrails in the GPT Tutor largely eliminate the negative exam effect, though no positive exam effect is observed |
description |
Guardrails in the GPT Tutor largely eliminate the negative exam effect, though no positive exam effect is observed |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2025 |
source_checked |
2026-10-18 |
generated |
by process:wiki-ingest on 2026-10-12 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — bastani-hamsa-2025 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.