On fully correct responses, the GPT few-shot models outperformed the fine-tuned models, averaging less than one category distance from the gold score¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r1 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? GPT models in all few-shot scenarios, including two-shot, performed very well on fully or almost fully correct answers (score 9-10), with average distance under one category from gold. → Abigail Victoria Gurin Schleifer 2026 q2 i? On the fully correct responses, the GPT models outperformed the fine-tuned models on both items. → Abigail Victoria Gurin Schleifer 2026 (2)

Evidence¶

Abigail Victoria Gurin Schleifer 2026¶

Abigail Victoria Gurin Schleifer, Moriah Ariely, Beata Beigman Klebanov, Asaf Salman, Giora Alexandron. (2026). Quality-Conditioned Agreement in Automated Short Answer Scoring: Mid-Range Degradation and the Impact of Task-Specific Adaptation. arXiv. https://arxiv.org/abs/2605.07647 doi:10.48550/arxiv.2605.07647

q2 · i? · causal · r1

Score-level analysis (Results, Table 1): GPT models "performed very well on fully or almost fully correct answers (score 9-10)", with "average distance of less than one category from the gold score".

"the GPT models in all few-shot scenarios – including those with only two examples – performed very well on fully or almost fully correct answers (score 9-10), showing average distance of less than one category from the gold score."

Abigail Victoria Gurin Schleifer 2026 (2)¶

Abigail Victoria Gurin Schleifer, Moriah Ariely, Beata Beigman Klebanov, Asaf Salman, Giora Alexandron. (2026). Quality-Conditioned Agreement in Automated Short Answer Scoring: Mid-Range Degradation and the Impact of Task-Specific Adaptation. arXiv. https://arxiv.org/abs/2605.07647

q2 · i? · design · r2

Comparison on fully correct responses (Results): "the GPT models outperformed the fine-tuned models on both items".

"On the fully correct responses, the GPT models outperformed the fine-tuned models on both items."

Discussion¶

Related Claims¶

• Ai Mid Range Scoring Degradation U Shape
• Mid-range degradation is conditioned on task-specific adaptation: it is most severe for few-shot LLMs with few examples, decreases with more task-specific data, and is mildest for the fine-tuned encoder — related
• Claude's few-shot scoring behaved erratically across items: best few-shot model on Item 1 but by far the worst model on Item 2 — related