Edu-QuRating pipeline and Edu-QuRater scorers¶
Product or Programme · All products and programmes
Evidence · 3 claims (2 for, 1 mixed) · 1 study (1 design),q2· 0 of 1 report an effect size · 3 claims rest on one study
Description¶
A software and trained-model package developed by Garrod et al. that uses rubric-guided LLM preference supervision to score and curate educational text for corpus filtering, language-model training, and educational-material support.
Components¶
- Edu-QuRating pipeline and released Edu-QuRater scorers, rubrics, code, and models: Edu-QuRating is a data-curation pipeline that samples text pairs from an education-rich corpus, obtains soft LLM-judge preference labels under criterion-specific rubrics, and distils them into single-text Edu-QuRater score layers with a neural Bradley–Terry objective. The core model produces six criterion-specific scores per text, with additional student- and teacher-facing foundational-literacy scorers. The article releases rubrics, source code, a nanotron fork, and trained models on Hugging Face, stating the scorers are "deployed for corpus filtering, small-language-model pre-training mixtures, and response-level GRPO rewards." (Garrod et al. (2026))
Claims¶
- Edu-QuRaters recover held-out GPT-4.1-mini pairwise educational preferences from single-text scores, with mean accuracy rising from 0.895 (Sheared-LLaMA-1.3B) to 0.917 (Gemma-3-4B-PT) [+M]
- In held-out pairwise judge evaluation, combining Edu-QuRater and answer-structure GRPO rewards yields the highest pedagogical-quality win rate (81.08%) while retaining instruction-following above parity (68.24%) against the Qwen3-4B base model [+W]
- Edu-QuRater rewards alone raise pedagogical-quality win rate (77.70%) but fall below parity on instruction following (40.54%), consistent with over-optimizing a proxy reward [~W]
Related Products and Programmes¶
-
Key Sources¶
- Garrod, O. G. B., Ince, R. A. A., Liu, M., Huti, M., Boos, M., Waldock, A., Andrews, D., Alonso-Kropil, R., & Atherton, P. (2026). Edu-QuRating: Multi-Dimensional Educational Data Curation with Distilled Pairwise Judgements. arXiv. https://arxiv.org/abs/2609.09425. doi:10.48550/arxiv.2609.09425
Page metadata
| Field | Value |
|---|---|
type |
product |
id |
edu-qurating-pipeline-and-edu-qurater-scorers |
title |
Edu-QuRating pipeline and Edu-QuRater scorers |
description |
A software and trained-model package developed by Garrod et al. |
status |
review |
product_kind |
software |
generated |
by process:settle-candidates on 2026-10-10 |
trust |
unverified — no one has confirmed this page's content |
Frontmatter is the page's machine-readable record — see the Schema & Guide.