ChatGPT's exam scores were within 10% of human grades 70% of the time in a study of AI-based grading¶
Claim · All claims
Evidence · 1 study · associationalr?·q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? In Flodén's (2025) study, ChatGPT's exam scores were within 10% of human grades 70% of the time, and instructors were surprised at how closely AI's assessments matched their own. → floden
Evidence¶
floden¶
Jacob, S., Miceli, H., & Schneider, H. (2026, May). Landscape of emerging technologies in higher education: A review. Digital Promise & The American Association of Colleges & Universities. https://doi.org/10.51388/20.500.12265/293
q2 · i? · associational · r?
The review reports, citing Flodén (2025), that AI-based grading systems showed "reasonably reliable performance compared to human graders," with ChatGPT's exam scores "within 10% of human grades 70% of the time." These are descriptive agreement figures, not effect sizes.
"In Flodén's (2025) study, ChatGPT's exam scores were within 10% of human grades 70% of the time, and instructors were surprised at how closely AI's assessments matched their own."
Discussion¶
Related Claims¶
- ChatGPT zero-shot relevance extraction for instruction-quality assessment retrieves mostly irrelevant utterances and overestimates instruction quality — related
- Unrestricted ChatGPT access yields better practice performance but significantly worse exam scores — related
- Hint-based prompts raise ChatGPT scoring agreement with human experts to 98–100%, while no-hint prompts are unstable (Q2 precision 0.42) — related
- ChatGPT-3.5 feedback on EAP argumentative essays aligns closely with human instructor scoring and identifies surface-level issues, but is less effective on rhetorical structure and contextual nuance — related
- Claude3 and PaLM2 show substantial agreement but fall short on identifying failing grades — related
- LLM grading performance relative to human graders is mixed: comparable in 12 reviewed cases, worse in 11, with the strongest alignment on short, straightforward tasks — related
- Agentic assessment systems report grading time reductions of 50–70% with 90–95% agreement with human graders — related
- ChatGPT in thinking mode shows the strongest agreement with human examiners on essay-style exam grading (weighted kappa 0.718) — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
chatgpt-grading-within-10-percent-human |
title |
ChatGPT's exam scores were within 10% of human grades 70% of the time in a study of AI-based grading |
description |
ChatGPT's exam scores were within 10% of human grades 70% of the time in a study of AI-based grading |
status |
review |
evidence_strength |
weak |
newest_evidence |
2026 |
recheck_by |
2029 |
generated |
by process:wiki-ingest on 2026-10-09 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — floden |
Frontmatter is the page's machine-readable record — see the Schema & Guide.