Chatbot access during mathematics practice improved practice performance but lowered subsequent unaided exam performance, and a hint-giving interface eliminated the penalty¶

Claim · All claims
Evidence · 1 study (2 entries) · theoretical r3 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? In a field experiment with nearly a thousand secondary mathematics students, standard chatbot access improved practice performance yet lowered unaided exam performance relative to controls. → Daiki Nakamura 2026 q2 i? Within the same experiment, the same model produced harm under an answer-giving interface and no harm under a hint-giving one. → Daiki Nakamura 2026 (2)

Evidence¶

Daiki Nakamura 2026¶

Daiki Nakamura. (2026). Augmentation or Erosion? Regulating Cognitive Offloading in Generative AI-Supported STEM Learning. Educational Psychology Review. https://doi.org/10.1007/s10648-026-10224-6

q2 · i? · theoretical · r3

The review reports, citing Bastani et al., a field experiment with nearly a thousand secondary mathematics students. Students treated the tool as a "crutch," reproducing its answers and errors, and believed they had learned as much as peers while performing worse on the unaided exam.

"access to a standard chatbot interface during practice dramatically improved performance on the practice problems themselves, yet lowered performance on a subsequent exam taken without AI relative to a control group that had never had access"

Daiki Nakamura 2026 (2)¶

Daiki Nakamura. (2026). Augmentation or Erosion? Regulating Cognitive Offloading in Generative AI-Supported STEM Learning. Educational Psychology Review. https://doi.org/10.1007/s10648-026-10224-6

q2 · i? · theoretical · r3

The review reports, citing Bastani et al., that the hint-based GPT Tutor withheld answers and offered teacher-crafted hints, keeping the target process with the student and eliminating the learning penalty the answer-giving interface produced.

"Within a single experiment, the same model produced harm under an answer-giving interface and no harm under a hint-giving one (Bastani et al., 2025)"

Discussion¶

Related Claims¶

• Reviews citing one randomised trial (Bastani et al. 2025) report that unrestricted GPT-4 access in high-school mathematics raised assisted practice performance but left students worse off after withdrawal — possibly the same claim (merge candidate)
• Unfettered GPT-4 access during practice harms subsequent unassisted exam performance in high school mathematics — a narrower finding that bears on this claim
• Guardrails in the GPT Tutor largely eliminate the negative exam effect, though no positive exam effect is observed — a narrower finding that bears on this claim
• Access to a standard GPT-4 chat tutor during practice substantially improves practice performance in high school mathematics — a narrower finding that bears on this claim
• A guardrailed GPT-4 tutor with teacher-designed hints improves assisted practice performance more than a standard GPT-4 interface — related
• Students do not perceive the learning harm from copying GPT solutions and overestimate their exam performance — related
• Students use standard GPT-4 access as a crutch—copying answers rather than engaging with the material—which explains the harm to learning — related
• Greater answer offloading during learning is associated with lower subsequent unaided performance (exploratory) — related