Simulated Human-in-the-Loop workflow reduces required full human scoring of written productions by at least 50%¶

Claim · All claims
Evidence · 1 study · design r2 · q1
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q1 i? Based on observed AI performance and candidate distributions in recent editions, the proposed process would reduce by at least 50% the number of written productions requiring full human scoring. → Curi 2026

Evidence¶

Curi 2026¶

Curi, M. E., Capdehourat, G., Amigo, I., Romano, M., Serra, R., Silveira, A., Peri, A. (2026). A Human-in-the-Loop Framework for AI-Assisted Scoring in Large-Scale Writing Assessment. arXiv. https://arxiv.org/abs/2609.05143 doi:10.48550/arxiv.2609.05143

q1 · i? · design · r2

Simulation of the proposed Human-in-the-Loop framework using data from the 2024 and 2025 editions, not an operational deployment. The article reports the process "would reduce by at least 50% the number of written productions requiring full human scoring."

"Based on data from the most recent test editions and the AI model performance observed in this study, the proposed process would reduce by at least 50% the number of written productions requiring full human scoring."

Discussion¶

Related Claims¶

• A hybrid human-AI workflow using GPT-4o with retrieval-augmented generation supported efficient inductive thematic analysis while preserving researcher judgment — related
• Respondents proposed testing and monitoring (61.9%) and human involvement (22.7%) as leading mitigations for algorithmic bias — related
• The revised Round II grading workflow reduced AI over-awarding and total-score MAD, most clearly for OE1 (7.1% to 4.8%) and QM (9.4% to 3.8%) — related
• Human scoring and feedback for HDR takes about 11 minutes 30 seconds per learner, while GPT-based scoring and feedback is nearly instant — related