Self-referential analytic assessment framework¶
Research Method · All research methods
Evidence · 3 claims (3 for) · 1 study (1 design),q2· 0 of 1 report an effect size · 3 claims rest on one study
Description¶
An evaluation paradigm for analytic essay scoring that shifts from normative (inter-learner ranking) to within-learner interpretation of analytic profiles. "Self-referential analytic assessment focuses on intra-learner variability, identifying aspects where a student performs significantly better or worse than their average proficiency." Standardised analytic scores are averaged within a learner, and deviations of each aspect from that average define a dual binary classification task for positive and negative feedback, using a one-standard-deviation threshold. The article positions this as a stricter, more pedagogically relevant test of analytic assessment capability and the first such framework for L2 AES.
Accounts¶
- Self-referential analytic assessment framework: An evaluation paradigm for analytic essay scoring that shifts from normative (inter-learner ranking) to within-learner interpretation of analytic profiles. "Self-referential analytic assessment focuses on intra-learner variability, identifying aspects where a student performs significantly better or worse than their average proficiency." Standardised analytic scores are averaged within a learner, and deviations of each aspect from that average define a dual binary classification task for positive and negative feedback, using a one-standard-deviation threshold. The article positions this as a stricter, more pedagogically relevant test of analytic assessment capability and the first such framework for L2 AES. (Stefano Bannò et al. (2026))
Claims¶
- Rank-based metrics can be misleading for analytic assessment: systems prompted for holistic scoring achieve strong, sometimes stronger, correlations with analytic scores than when prompted for those aspects [+M]
- In a self-referential framework, LLMs, particularly GPT-4.1, tend to outperform a single operational human rater at identifying relative weaknesses (negative feedback) across several Language and Content aspects [+M]
- Human operational raters remain stronger than LLMs at identifying relative strengths (positive feedback) within learner profiles [+M]
Related Research Methods¶
-
Key Sources¶
- Stefano Bannò, Kate Knill, Mark Gales. (2026). Towards Self-Referential Analytic Assessment: A Profile-Based Approach to L2 Writing Evaluation with LLMs. https://arxiv.org/abs/2605.04298
Page metadata
| Field | Value |
|---|---|
type |
research-method |
id |
self-referential-analytic-assessment-framework |
title |
Self-referential analytic assessment framework |
description |
An evaluation paradigm for analytic essay scoring that shifts from normative (inter-learner ranking) to within-learner interpretation of analytic profiles. |
status |
review |
generated |
by process:settle-candidates on 2026-10-11 |
trust |
unverified — no one has confirmed this page's content |
Frontmatter is the page's machine-readable record — see the Schema & Guide.