Newer LLM generations such as GPT-4 generally outperform GPT-3.5 in short-answer grading accuracy, but this advantage is not consistent across all tasks and domains¶

Claim · All claims
Evidence · 1 study · review r3 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? Initial results indicate GPT-4 generally outperforms GPT-3.5 in short-answer grading accuracy, though the advantage is not yet consistent across tasks and domains. → Marcin Jukiewicz and Michał Wyrwa 2026

Evidence¶

Marcin Jukiewicz and Michał Wyrwa 2026¶

Marcin Jukiewicz and Michał Wyrwa. (2026). Can ChatGPT Replace the Teacher in Assessment? A Review of Research on the Use of Large Language Models in Grading and Providing Feedback. Appl. Sci. 2026, 16, 680. https://doi.org/10.3390/app16020680

q2 · i? · review · r3

The review's RQ4 synthesis notes only a handful of studies allow tracking model generations, and reports the GPT-4-over-GPT-3.5 advantage as initial and task-limited. No effect size is printed.

"Initial results indicate that, for instance, GPT-4 generally outperforms GPT-3.5 in the accuracy of short-answer grading; however, this advantage is not yet consistent across all tasks and domains"

Discussion¶

Related Claims¶

• LLM grading performance relative to human graders is mixed: comparable in 12 reviewed cases, worse in 11, with the strongest alignment on short, straightforward tasks — the general claim this finding bears on, with the studies behind it
• Newer LLM generations do not consistently agree better with human essay graders — reports the opposite
• Model selection experiments show newer models are not strict improvements, with component-specific effects on quality metrics — related
• GPT-4o and GPT-5.2 achieve higher accuracy and more fully correct answers on university IVDA multiple-choice questions than the smaller Mistral-7B and DeepSeek-R1 models — related