GradeOpt¶
Product or Programme · All products and programmes
Evidence · 5 claims (2 for, 3 mixed) · 2 studies (1 causal, 1 design),q2· 0 of 2 report an effect size · 5 claims rest on one study
Description¶
GradeOpt is a multi-agent LLM coding framework developed by Copur-Gencturk et al. that automates rubric-based coding of open-ended responses through Grader, Reflector, and Refiner agents.
Components¶
Claims¶
- The multi-agent LLM GradeOpt achieved substantial agreement with human coders on overall PCK items (κ = .68, QWK = .79) and outperformed RoBERTa, SBERT, and naïve GPT-4o prompting [+M]
- On CK items, GradeOpt showed near-perfect alignment with human codes (κ = .88 and .85; QWK = .89 and .94), comparable to human–human agreement [+M]
- Gaps between Cohen's κ and QWK indicate GradeOpt often assigned codes adjacent to human codes rather than entirely discrepant ones [~W]
- All automated models, including GradeOpt, were most challenged by knowledge of teaching mathematics items (GradeOpt κ = .59) and least challenged by knowledge of students' mathematical thinking (κ = .79) [~W]
- Grading recipes without calibration examples perform substantially worse on MAE, though multi-step without examples ranks second on QWK [~W]
Related Products and Programmes¶
-
Key Sources¶
- Copur-Gencturk, Y., Moreno, K., Chu, Y., Li, H., & Tang, J. (2026). Automated coding of content and pedagogical content knowledge of mathematics using a multi-agent large language model. ZDM – Mathematics Education. https://doi.org/10.1007/s11858-026-01796-2
Page metadata
| Field | Value |
|---|---|
type |
product |
id |
gradeopt |
title |
GradeOpt |
description |
GradeOpt is a multi-agent LLM coding framework developed by Copur-Gencturk et al. |
status |
review |
product_kind |
software |
generated |
by process:settle-candidates on 2026-10-11 |
trust |
unverified — no one has confirmed this page's content |
Frontmatter is the page's machine-readable record — see the Schema & Guide.