For individual students, GPT-4 grades match instructor grades more closely than peer grades do, with smaller dispersion¶
Claim · All claims
Evidence · 1 study · causalr1·q2
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person
Design limits · Chris Impey 2024: the intervention came bundled with another change (read from the study by two models, 2026-10-27)
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? Per-student mean instructor-minus-GPT-4 scores were smaller in magnitude with smaller dispersion than instructor-minus-peer scores in all three courses (e.g., -0.01 ± 0.15 vs -0.07 ± 0.26 in astronomy). → Chris Impey 2024
Evidence¶
Chris Impey 2024¶
Chris Impey, Matthew Wenger, Nikhil Garuda, Shahriar Golchin, Sarah Stamer. (2024). Using Large Language Models for Automated Grading of Student Writing about Science. Preprint. https://doi.org/10.21203/rs.3.rs-3962175/v1
q2 · i? · causal · r1
Individual-student comparison across all twelve assignments for ten students per course (Figure 1); the article reports analogous results for astrobiology (-0.02 ± 0.17 vs -0.09 ± 0.20) and history and philosophy (-0.09 ± 0.24 vs -0.23 ± 0.30), concluding GPT-4 performs better than peer grading.
"For the introductory course, the mean instructor minus GPT-4 score is -0.01 ± 0.15, compared to the mean instructor minus peer grading score of -0.07 ± 0.26. The LLM grade is very close to the instructor grade, with a smaller dispersion than the peer grade."
Discussion¶
Related Claims¶
- In three astronomy MOOCs, GPT-3.5 and GPT-4 grades are closer to instructor grades with lower dispersion than peer grades — a broader claim this one bears on
- Peer graders in these MOOCs score more leniently and with greater dispersion than the instructor — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
gpt4-individual-student-grading-beats-peer-grading |
title |
For individual students, GPT-4 grades match instructor grades more closely than peer grades do, with smaller dispersion |
description |
For individual students, GPT-4 grades match instructor grades more closely than peer grades do, with smaller dispersion |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2024 |
generated |
by process:wiki-ingest on 2026-10-27 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — chris-impey-2024 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.