AI access raises unaided essay quality in both sessions, while AI-generated text detectable in Session One essays disappears by Session Two¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r3 · q4
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q4 i? Essays written during AI access score 0.34 SD higher in overall quality, and essays written unaided one week later are still 0.24 SD higher, indicating a learning channel beyond AI-generated text. → Zara Contractor and Germán Reyes 2026 q4 i? The share of treated students' Session One text flagged as AI-generated rises by 22.2 pp, and this detection effect collapses to 0.7 pp in the unaided Session Two essays. → Zara Contractor and Germán Reyes 2026 (2)

Evidence¶

Zara Contractor and Germán Reyes 2026¶

Zara Contractor and Germán Reyes. (2026). Experimental Evidence on the Learning Impact of Generative AI. https://arxiv.org/abs/2607.08849 doi:10.48550/arxiv.2607.08849

q4 · i? · causal · r3

Essays graded blind by three or four master's/PhD-level Prolific graders per essay (309 recruited), averaged with LLM ratings. The article reports the Session Two effect as 0.33 points (0.24 SD, p= 0.083), with the largest Session Two dimension effects on use of evidence and examples (0.35 SD) and writing style and clarity (0.27 SD).

"In Session One, treated essays score ˆβ= 0.45points higher in overall quality (or0.34SD,p= 0.024), an 8 percent increase over the control mean of 5.50 points"

Zara Contractor and Germán Reyes 2026 (2)¶

Zara Contractor and Germán Reyes. (2026). Experimental Evidence on the Learning Impact of Generative AI. https://arxiv.org/abs/2607.08849

q4 · i? · causal · r3

Pangram 4 detector applied to essays from the same randomized experiment. The article reports the TOT estimate as 31.8 pp and that by Session Two "The treatment effect collapses from ˆβ= 22.2pp to ˆβ= 0.7pp (p= 0.494)", with no plagiarism-flag differences in either session.

"The fraction of text classified as AI-generated rises byˆβ= 22.2pp in the AI-allowed group (p <0.001), a 337 percent increase over the control mean of 6.6 percent."

Discussion¶

Related Claims¶

• AI access raises immediate unaided test scores by 7.2 percentage points (0.28 SD) after a timed learning phase — related
• One week later, the AI-access group's lead on unaided assessments shrinks to 3.9 pp (0.21 SD), short of statistical significance (p = 0.068) — related
• Learning outcomes differ by how students use AI: augmentation and mixed users show gains in both sessions, while automation users' Session One essay advantage is absent from their unaided essays — related
• Independent evaluation found commercial AI-text detectors neither sufficiently accurate nor sufficiently reliable for high-stakes use — related
• In a documented exploratory audit, one human-authored manuscript received materially different classifications from five commercial detectors, spanning 0% human to Human Generated — related
• AI-text detectors misclassify authentic TOEFL essays by non-native English speakers at a mean false-positive rate of 61.3% across seven detectors, far more often than native-speaker essays — related
• The availability of AI assistants improved the quality of submitted admissions essays, particularly mechanical aspects and especially for international applicants — related
• Admissions officers can often discriminate AI-written from human-written essays, achieving an AUC of 0.70, well above chance but below commercial detectors — related
• Access to a generative-AI assistant raises task performance for both lower- and higher-education adults, with significantly larger gains for lower-education participants — related
• Access to expert-verified AI study materials is associated with a 2.34-mark difference-in-differences advantage on a 50-mark examination component, reflecting protection from a decline rather than a rise — related
• Prior genAI use and incentives show no clear effects on later memory retention — related
• Intervention studies reported larger effect sizes than cross-sectional studies, and studies including more teaching assistants reported larger effects — related