LLM feedback receives higher expert ratings on six of seven quality dimensions, with tone the exception¶
Claim · All claims
Evidence · 1 study · associationalr2·q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? For sentence-level feedback, LLM feedback scored higher than human feedback on six of seven dimensions, with the largest gap in specificity (+1.85) and the only exception tone (−0.19). → Yijun Liu 2026
Evidence¶
Yijun Liu 2026¶
Yijun Liu, Yifan Song, John Gallagher, Sarah Sterman, Tal August. (2026). FOXGLOVE: Understanding Goal-Oriented and Anchored Writing Feedback from Experts and LLMs on Argumentative Essays. https://arxiv.org/abs/2606.06271 doi:10.48550/arxiv.2606.06271
q2 · i? · associational · r2
Expert quality-rating study (§5.1): two blind college-level English instructors rated 105 comments (61 LLM, 44 human) on 5-point scales across seven dimensions, yielding 1,430 ratings with Cohen's κw = 0.36 for sentence-level feedback. The article reports "the highest difference being in specificity (∆ = +1.85)".
"For sentence level feedback, LLM feed- back scored higher than human feedback on six out of seven dimensions: with the highest difference being in specificity (∆ = +1.85), followed by ac- tionability (+1.40). The only dimension models did not perform better on was tone (−0.19)."
Discussion¶
Related Claims¶
- Instructors use more first-person and second-person pronouns and more questions than LLMs in feedback comments — related
- A substantial part of the LLM rating advantage appears attributable to comment length, as length-adjusted scores converge on every dimension except tone — related
- Insta-Reviewer generates near-human-quality feedback on student project reports, outperforming expert feedback on Problems and Positive Tone but trailing it on Readability, Suggestions, and Factuality — related
- LLMs and instructors agree broadly on where feedback belongs but diverge on exact sentence spans, with span overlap strongly goal-dependent and LLMs highlighting more of each essay — related
- Children's plan quality improves substantially after receiving LLM-generated adaptive feedback — related
- Expert raters rate TeachingCoach higher than a GPT-4o baseline on clarity, tone, reflection, and acknowledgment of user input — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
llm-feedback-higher-expert-ratings |
title |
LLM feedback receives higher expert ratings on six of seven quality dimensions, with tone the exception |
description |
LLM feedback receives higher expert ratings on six of seven quality dimensions, with tone the exception |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2026 |
recheck_by |
2029 |
generated |
by process:wiki-ingest on 2026-10-10 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — yijun-liu-2026 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.