ASAG and AEG systems differ on answer length, grading focus, and question openness¶
Claim · All claims
Evidence · 1 study · reviewr2·q2
Currency · newest study 2015 · not yet checked against its sources · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? ASAG systems grade content-focused responses to objective, close-ended questions, whereas AEG systems grade style-focused responses to subjective, open-ended questions. → Steven Burrows 2015
Evidence¶
Steven Burrows 2015¶
Steven Burrows, Iryna Gurevych, Benno Stein. (2015). The Eras and Trends of Automatic Short Answer Grading. Int J Artif Intell Educ. https://doi.org/10.1007/s40593-014-0026-8
q2 · i? · review · r2
Narrative review synthesis in the Question Type section, supported by cited ETS systems c-rater and e-rater as examples of ASAG and AEG respectively. The review reports that ASAG focuses on content while AEG focuses on style, with short answers ranging from one phrase to one paragraph. No effect size is printed.
"Specifically, ASAG systems require answers to objective or close-ended questions. In contrast, AEG systems require answers to subjective or open-ended questions (Leacock and Chodorow 2003; Siddiqi and Harrison 2008b; Wood et al. 2006)."
Discussion¶
Related Claims¶
- LLM grading performance is markedly worse for open-ended history and philosophy questions than for content-based astronomy and astrobiology questions — related
- LLMs perform worse on complex, open-ended, or subjective assignments than on short, clearly formulated answers — a broader claim this one bears on
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
asag-content-focus-versus-aeg-style |
title |
ASAG and AEG systems differ on answer length, grading focus, and question openness |
description |
ASAG and AEG systems differ on answer length, grading focus, and question openness |
status |
review |
evidence_strength |
weak |
newest_evidence |
2015 |
generated |
by process:wiki-ingest on 2026-10-27 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — steven-burrows-2015 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.