RACES configurations improve QWK essay-scoring agreement over evaluated pretrained baselines on ASAP 2.0, with LLaMA2-7B-based configurations slightly outperforming Qwen2.5-Instruct-7B-based ones¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? After PPO optimization, all four evaluated policy–reward configurations achieved competitive QWK, with LLaMA2-7B-based configurations slightly outperforming Qwen2.5-Instruct-7B-based ones and Mistral-7B reward models generally stronger than GPT2-XL. → Zhenxin Zhang 2026

Evidence¶

Zhenxin Zhang 2026¶

Zhenxin Zhang, Ziyu Ding, Mengyun Liu and Haiwei Sang. (2026). RACES: reward-aligned consistent essay scoring with large language models. International Journal of Educational Technology in Higher Education. https://doi.org/10.1186/s41239-026-00607-8

q2 · i? · design · r2

Numerical evaluation shown in Fig. 5a of four policy–reward configurations on the ASAP 2.0 benchmark (QWK reported as percentages, e.g. 82.7 for LLaMA2-7B with Mistral-7B reward model vs 71.6 pretrained). The article reports "all four configurations achieve competitive performance under the evaluated setting". No confidence intervals or significance tests were reported.

"As shown, all four configurations achieve competitive performance under the evaluated setting. The LLaMA2-7B-based configurations slightly outperform the Qwen2.5-Instruct-7B-based configurations, which may be related to differences in their adaptation to English argumentative essays"

Discussion¶

Related Claims¶

• Mistral-7B reward model outperforms GPT2-XL on Hit@k ranking and Pairwise Accuracy for proxy preference discrimination, at higher computational cost — related
• After PPO optimization, auxiliary feedback semantic similarity (SimCSE) reaches 81.3% and 79.3% for the two policy models, higher than evaluated pretrained baselines — related
• The PPO-driven policy substantially outperformed a random policy baseline, with cumulative reward of 95.12 ± 1.23 versus 12.3 ± 4.7 — related