AI-generated 10-question exams provided higher maximum test information and reliability (Imax = 3.85, R = 0.79) than the standardized exams (Imax = 2.61, R = 0.72)¶
Claim · All claims
Evidence · 1 study · associationalr2·q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person · recheck by 2028: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? Averaged over class exams, AI-generated exams achieved higher maximum test information and conditional reliability than standardized-test-composed exams. → Calvin Isley 2025
Evidence¶
Calvin Isley 2025¶
Calvin Isley, Joshua Gilbert, Evangelos Kassos, Michaela Kocher, Allen Nie, Emma Brunskill, Ben Domingue, Jake Hofman, Joscha Legewie, Teddy Svoronos, Charlotte Tuminelli, Sharad Goel. (2025). Assessing the Quality of AI-Generated Exams: A Large-Scale Field Study. https://arxiv.org/abs/2508.08314 doi:10.48550/arxiv.2508.08314
q2 · i? · associational · r2
Test information curves computed for every class exam in the field study, shown in Figure 5 and split by exam type; the AI exams reached "Imax = 3.85" against the standardized exams' Imax = 2.61 and R(θ) = 0.72.
"The maxi- mum test information for the AI-generated exams, averaged over exams, is Imax = 3.85, corresponding to a reliabil- ity of R(θ) = 0 .79."
Discussion¶
Related Claims¶
- AI-generated exam questions were somewhat easier than standardized-test questions, with average difficulty βAI = −0.45 versus βSTD = 0.35 — related
- AI-generated questions showed slightly higher average discrimination than standardized test questions (αAI = 1.3 vs αSTD = 1.2), with moderate evidence — related
- For all θ < 0.75, AI-generated exams were on average more informative than the standardized exams, with I(0) = 3.36 versus 2.5 — related
- A larger share of AI-generated questions were highly or very highly discriminating (36%) compared to standardized test questions (21%) — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
ai-exams-higher-test-information |
title |
AI-generated 10-question exams provided higher maximum test information and reliability (Imax = 3.85, R = 0.79) than the standardized exams (Imax = 2.61, R = 0.72) |
description |
AI-generated 10-question exams provided higher maximum test information and reliability (Imax = 3.85, R = 0.79) than the standardized exams (Imax = 2.61, R = 0.72) |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2025 |
recheck_by |
2028 |
generated |
by process:wiki-ingest on 2026-10-10 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — calvin-isley-2025 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.