In math problem solving, a tutor LLM alone significantly improved unassisted test accuracy over no-agent control, while peer-like error agents alone did not reach significance¶
Claim · All claims
Evidence · 1 study · causalr?·q3
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q3 i? The tutor effect over Control was significant (χ²(1)=8.27, p=.004), while the peers-only effect did not reach significance (χ²(1)=0.88, p=.347). → Harsh Kumar 2026
Evidence¶
Harsh Kumar 2026¶
Harsh Kumar, Zi Kang (Jace) Mu, Jonathan Vincentius, Ashton Anderson. (2026). Human Thinking under Plural LLM Assistance: Mathematical Problem Solving and Open-Ended Writing. Working Paper, July 2026, Toronto, Canada. https://arxiv.org/abs/2604.02677 doi:10.48550/arxiv.2604.02677
q3 · i? · causal · r?
Exploratory pairwise chi-square tests on unassisted test accuracy in the pre-registered Experiment-1 (315 analyzed participants), comparing each agent arrangement against the no-agent Control condition; the quote reports both printed test statistics and p-values.
"the tutor effect was significant (Control vs. Tutor Only: 𝜒 2(1)= 8.27,𝑝=. 004), while the peers effect alone did not reach significance (Control vs. Peers Only: 𝜒 2(1)= 0.88,𝑝=. 347)."
Discussion¶
Related Claims¶
- In math problem solving, a tutor-plus-peers LLM arrangement produced the strongest unassisted post-task test accuracy, with accuracy increasing monotonically from Control to Tutor + Peers — related
- Purely peer groups working purposefully out-of-reach problems did not fully succeed but still made significant progress, internalizable by individual members — related
- AI access raises immediate unaided test scores by 7.2 percentage points (0.28 SD) after a timed learning phase — related
- In a randomized experiment on fraction problems, participants who used an AI assistant solved fewer unassisted test problems and skipped more than control participants — related
- Reviews citing one randomised trial (Bastani et al. 2025) report that unrestricted GPT-4 access in high-school mathematics raised assisted practice performance but left students worse off after withdrawal — related
- Conversational-agent prompts improve student dialogue outcomes at both individual and dyad levels — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
tutor-alone-significant-peers-alone-not-math |
title |
In math problem solving, a tutor LLM alone significantly improved unassisted test accuracy over no-agent control, while peer-like error agents alone did not reach significance |
description |
In math problem solving, a tutor LLM alone significantly improved unassisted test accuracy over no-agent control, while peer-like error agents alone did not reach significance |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2026 |
generated |
by process:wiki-ingest on 2026-10-11 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — harsh-kumar-2026 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.