In one study of five LLMs scoring 60 undergraduate essays on IELTS rubric dimensions, systematic underestimation of human scores was the dominant bias trend, with some models also compressing scores into a narrow range¶

Claim · All claims
Evidence · 1 study · associational r1 · q2
Currency · newest study 2026 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? Of the five LLMs tested on 60 undergraduate essays scored against the IELTS Task 2 rubric, most systematically assigned lower scores than human raters, a consistent negative bias the article interprets as a calibration offset rather than random fluctuation. → Tongxi Liu 2026

Evidence¶

Tongxi Liu 2026¶

Tongxi Liu, Luyao Ye, Wei Yan. (2026). A framework for evaluation of large language models in essay assessment: Reliability, alignment, and causal reasoning. Computers and Education: Artificial Intelligence. https://doi.org/10.1016/j.caeai.2026.100565

q2 · i? · associational · r1

Bland-Altman bias analysis across five LLMs (GPT-4.1, Llama 4 Maverick, Gemini 2.5 Flash, Claude Sonnet 4, DeepSeek R1) scoring 60 essays written by non-native English-speaking students for an undergraduate course, on the four IELTS Task 2 rubric dimensions, reported in Results §4.3 with Fig. 4. The article describes the negative bias as a "general calibration offset rather than random fluctuation" with magnitude varying by model and criterion.

"Specifically, underestimation emerged as the dominant trend, indi - cating that most models systematically assigned lower scores than human raters (Fig. 4)."

Discussion¶

Related Claims¶

• In one reviewed study, GPT-4 automated essay scoring showed score inflation, with AI–human rating discrepancies exceeding 1.5 standard deviations in 38% of graduate-level bibliographies — reports the opposite
• Under the initial rubric with direct evaluation, machine raters systematically underestimate openness while relevance and clarity align near-perfectly — related
• Small language models used as automated judges exhibit severe leniency bias against tutoring responses — reports the opposite
• ML-based scoring approaches more often overestimated expert-assigned scores, whereas LLM-based approaches more often underestimated them — possibly the same claim (merge candidate)
• Automated judging aligns strongly with three domain experts (r = 0.82 for role fidelity) but exhibits a conservative bias, scoring ethical deviation 0.08 points lower than humans — related
• Under LLM evaluators, structured rubrics produced larger negative deviations from the human baseline than free-form rubrics, the opposite direction of the human-evaluator pattern
• AI scoring shows a systematic conservative bias, under-grading writing so 15.3% (2024) and 16.5% (2025) of human-passing tests would fail under AI scoring — a narrower finding that bears on this claim
• Lower linguistic quality of physics explanations is associated with higher odds that AI-based scoring underestimates conceptual understanding, across all eleven examined AI-based scoring approaches — related
• LLMs overrate short underdeveloped essays and over-penalize minor language errors in well-developed essays — reports the opposite
• LLM grading performance relative to human graders is mixed: comparable in 12 reviewed cases, worse in 11, with the strongest alignment on short, straightforward tasks — related
• RAG prompting shows systematic leniency, over-scoring relative to teacher mean scores in all four rubric dimensions (total bias = 1.787) — reports the opposite