AI grading recovered the same five-student International Physics Olympiad team as official human grading in both rounds, though with a different internal ranking¶
Claim · All claims
Evidence · 1 study · designr3·q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? In both grading rounds the AI top five for OE2 contained the same five students as the human top five, ranked in a different order; at k=10 both rounds recovered 9 of the human top 10 and at k=20, 19 of 20. → Praveen Pathak 2026
Evidence¶
Praveen Pathak 2026¶
Praveen Pathak, Siddharth Tiwary, Charudatt Kadolkar, Vijay Singh, David Rakestraw, Shirish Pathare, and Anwesh Mazumdar. (2026). Large-scale AI grading of handwritten physics assessments: Score agreement and Olympiad team selection outcomes. https://arxiv.org/abs/2608.20521 doi:10.48550/arxiv.2608.20521
q2 · i? · design · r3
Outcome-level check on the OE2 final selection camp (26 candidates, five exam components each, combined as 0.6T+0.4E). The article reports "the AI top five contained the same five students as the human top five"; no effect size is printed.
"In both RI and RII, the AI top five contained the same five students as the human top five, although the AI ranked them in a different order."
Discussion¶
Related Claims¶
- AI total scores correlate strongly with official human scores on handwritten physics assessments (r = 0.91–0.97 in Round I; 0.93–0.96 in Round II) — related
- The revised Round II grading workflow reduced AI over-awarding and total-score MAD, most clearly for OE1 (7.1% to 4.8%) and QM (9.4% to 3.8%) — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
ai-recovers-same-olympiad-team |
title |
AI grading recovered the same five-student International Physics Olympiad team as official human grading in both rounds, though with a different internal ranking |
description |
AI grading recovered the same five-student International Physics Olympiad team as official human grading in both rounds, though with a different internal ranking |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2026 |
recheck_by |
2029 |
generated |
by process:wiki-ingest on 2026-10-10 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — praveen-pathak-2026 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.