AI assistance impairs unassisted performance and persistence in reading comprehension, replicating the fraction-task effects in a second domain¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r2 · q3 · i2 medium
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i2 In reading comprehension, the AI condition solved fewer unassisted test problems than controls (Cohen's d = -0.42). → Grace Liu 2026 q3 i2 The AI condition also skipped more test problems than controls (Cohen's d = 0.42). → Grace Liu 2026 (2)

Evidence¶

Grace Liu 2026¶

Grace Liu, Brian Christian, Tsvetomira Dumbalska, Michiel A. Bakker, Rachit Dubey. (2026). AI Assistance Reduces Persistence and Hurts Independent Performance. Published as a conference paper at COLM 2026. https://arxiv.org/abs/2604.04721 doi:10.48550/arxiv.2604.04721

q3 · i2 · causal · r2

RCT (201 recruited, 168 analyzed) replicating the design with SAT reading-comprehension questions; AI condition used a GPT-5 sidebar for 5 problems, then 3 unassisted. Solve-rate impairment replicated, "Cohen's d=− 0.42".

"Participants in the AI condition had a lower solve rate (mean 0.76, s.d. 0.34) than participants in the control condition (mean 0.89, s.d. 0.25; t(166) =− 2.72, P= 0.007; Cohen's d=− 0.42, 95% CI [−0.73,−0.11])."

Grace Liu 2026 (2)¶

Grace Liu, Brian Christian, Tsvetomira Dumbalska, Michiel A. Bakker, Rachit Dubey. (2026). AI Assistance Reduces Persistence and Hurts Independent Performance. Published as a conference paper at COLM 2026. https://arxiv.org/abs/2604.04721

q3 · i2 · causal · r2

Same reading-comprehension RCT; answers given in under 5 seconds were also counted as skipped. The AI condition's higher skip rate, "Cohen's d= 0.42", suggests reduced persistence in reading tasks.

"also had a higher skip rate (mean 0.08, s.d. 0.21) than participants in the control condition (mean 0.01, s.d. 0.08; t(166) = 2.69, P= 0.008; Cohen's d= 0.42, 95% CI [0.11, 0.72])."

Discussion¶

Related Claims¶

• Experiment 2 replicates the unassisted-performance impairment with a pretest-based exclusion, but the aggregate skip-rate difference does not reach significance — related
• In a randomized experiment on fraction problems, participants who used an AI assistant solved fewer unassisted test problems and skipped more than control participants — a broader claim this one bears on
• AI assistance reduces persistence: persistence costs concentrate among learners who used AI for direct solutions, not hints — related
• Persistence and performance costs concentrate among participants who used the AI to obtain direct solutions, while hint-users resemble controls — related
• AI access raises immediate unaided test scores by 7.2 percentage points (0.28 SD) after a timed learning phase — reports the opposite
• In math problem solving, a tutor-plus-peers LLM arrangement produced the strongest unassisted post-task test accuracy, with accuracy increasing monotonically from Control to Tutor + Peers — related
• Reviews citing one randomised trial (Bastani et al. 2025) report that unrestricted GPT-4 access in high-school mathematics raised assisted practice performance but left students worse off after withdrawal — related