FOXGLOVE

Product or Programme · All products and programmes
Evidence · 8 claims (7 for, 1 mixed) · 3 studies (2 associational, 1 design), q2 · 0 of 3 report an effect size · 8 claims rest on one study

Description

FOXGLOVE is a parallel dataset of sentence-anchored, goal-labeled instructor and LLM writing feedback with urgency rankings and expert quality ratings, created from twelfth-grade argumentative essays by Yijun Liu and colleagues.

Components

Claims

• LLMs and instructors agree broadly on where feedback belongs but diverge on exact sentence spans, with span overlap strongly goal-dependent and LLMs highlighting more of each essay [+W]
• Instructors use more first-person and second-person pronouns and more questions than LLMs in feedback comments [+W]
• Feedback givers agree on exact urgency tiers only about a fifth of the time, with most disagreements off by a single tier [+W]
• LLM feedback receives higher expert ratings on six of seven quality dimensions, with tone the exception [+W]
• Human-LLM agreement on a complex multi-label codebook falls well below human-human agreement, while LLM-LLM agreement is comparable to human-human agreement [+W]
• A substantial part of the LLM rating advantage appears attributable to comment length, as length-adjusted scores converge on every dimension except tone [+W]
• Instructors and LLMs distribute feedback similarly across the five argumentative goals, with Claim receiving the most feedback from both [+W]
• The LLM judge agrees with human annotators at 87.7% (κ = 0.842) on consensus pairs, but severity-rating agreement is uneven and medium-tier boundaries are uncertain [~W]

Related Products and Programmes

• WrAFT (Writing Assessment and Feedback Tool)

Key Sources

• Yijun Liu, Yifan Song, John Gallagher, Sarah Sterman, Tal August. (2026). FOXGLOVE: Understanding Goal-Oriented and Anchored Writing Feedback from Experts and LLMs on Argumentative Essays. https://arxiv.org/abs/2606.06271

Page metadata

Field	Value
type	product
id	foxglove
title	FOXGLOVE
description	FOXGLOVE is a parallel dataset of sentence-anchored, goal-labeled instructor and LLM writing feedback with urgency rankings and expert quality ratings, created from twelfth-grade argumentative essays by Yijun Liu and colleagues.
status	review
product_kind	dataset
generated	by process:settle-candidates on 2026-10-10
trust	unverified — no one has confirmed this page's content

Frontmatter is the page's machine-readable record — see the Schema & Guide.