AI access raises immediate unaided test scores by 7.2 percentage points (0.28 SD) after a timed learning phase¶
Claim · All claims
Evidence · 1 study · causalr3·q4
Currency · newest study 2026 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q4 i? Undergraduates randomly allowed to use generative AI while learning an unfamiliar topic score 7.2 pp (0.28 SD) higher on an immediate unaided knowledge test than the AI-forbidden control group. → Zara Contractor and Germán Reyes 2026
Evidence¶
Zara Contractor and Germán Reyes 2026¶
Zara Contractor and Germán Reyes. (2026). Experimental Evidence on the Learning Impact of Generative AI. https://arxiv.org/abs/2607.08849 doi:10.48550/arxiv.2607.08849
q4 · i? · causal · r3
Intent-to-treat estimate from a proctored, in-person randomized experiment (211 undergraduates, Session One) in which the AI-allowed group could use any generative AI during a 35-minute learning phase and all students took the test without resources. The article reports "a 13 percent improvement over the control mean" and a standardized 0.28 SD effect (p= 0.021); the corresponding TOT estimate is 10.4 pp, or 0.41 SD.
"Control students’ fraction of questions answered correctly rises from 30.6 percent to 56.3 percent (p<0.001); AI access adds a further ˆβ= 7.2pp, a 13 percent improvement over the control mean."
Discussion¶
Related Claims¶
- AI access raises unaided essay quality in both sessions, while AI-generated text detectable in Session One essays disappears by Session Two — related
- One week later, the AI-access group's lead on unaided assessments shrinks to 3.9 pp (0.21 SD), short of statistical significance (p = 0.068) — related
- Review reports a field experiment in which GPT access improved supported practice but was followed by poorer unaided test performance, mitigated by a guarded tutor — related
- Review reports a randomized trial in which unrestricted ChatGPT study produced lower delayed retention than traditional methods — related
- AI assistance that substitutes for learning activities may improve immediate performance while impeding long-term skill development — a broader claim this one bears on
- Students expect AI to help their test scores, but control students overestimate the effect relative to the experimental estimate — related
- Access to a generative-AI assistant raises task performance for both lower- and higher-education adults, with significantly larger gains for lower-education participants — related
- Instruction type predicts post-test performance with a large effect size after controls (linear and quantile regression) — related
- AI assistance impairs unassisted performance and persistence in reading comprehension, replicating the fraction-task effects in a second domain — reports the opposite
- In a randomized experiment on fraction problems, participants who used an AI assistant solved fewer unassisted test problems and skipped more than control participants — reports the opposite
- Students show large immediate pre-to-post knowledge gains within a single LLM-supported learning session — related
- Among GPT-assisted students, proactive and critical engagement patterns are associated with higher exam scores while limited engagement shows little benefit relative to control — related
- In math problem solving, a tutor-plus-peers LLM arrangement produced the strongest unassisted post-task test accuracy, with accuracy increasing monotonically from Control to Tutor + Peers — a narrower finding that bears on this claim
- Both AI and human tutoring produce larger GRE learning gains than a no-tutoring control — a narrower finding that bears on this claim
- Immediate task success does not establish learning that persists without AI support — related
- Undergraduates learning IoT tasks in VR with ChatGPT-integrated feedback reached significantly higher cognitive levels on memory, understanding, and application tasks than peers receiving traditional VR feedback — related
- AI-assisted performance during skill-building does not transfer to subsequent unassisted performance — reports the opposite
- Written examination scores were low and confounded, providing a serious warning about students' unaided performance on foundational calculations — related
- AI-generated (GPT-4) feedback produced no greater linguistic progress than human tutor feedback among ENL students over six weeks — related
- LLM-assisted self-study yields higher post-intervention exam performance than traditional resources among early-career knowledge workers — related
- Access to a structured conversational AI tutor increased post-test content scores over a no-AI holdout with ambient consumer AI available — related
- In math problem solving, a tutor LLM alone significantly improved unassisted test accuracy over no-agent control, while peer-like error agents alone did not reach significance — related
- Students learn more with a research-based AI tutor than with in-class active learning on identical content (post-test scores) — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
genai-access-raises-immediate-test-scores |
title |
AI access raises immediate unaided test scores by 7.2 percentage points (0.28 SD) after a timed learning phase |
description |
AI access raises immediate unaided test scores by 7.2 percentage points (0.28 SD) after a timed learning phase |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2026 |
source_checked |
2026-10-20 |
generated |
by process:wiki-ingest on 2026-10-10 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — zara-contractor-and-germán-reyes-2026 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.