398-item evaluation dataset of teacher-requested geometric diagrams from Coteach¶
Product or Programme · All products and programmes
Evidence · 2 claims (1 for, 1 mixed) · 1 study (1 design),q2· 0 of 1 report an effect size · 2 claims rest on one study
Description¶
A dataset released by Vishal Kumar et al. containing 398 real teacher requests and generated geometric-diagram code instances from Coteach for evaluating the mathematical and spatial soundness of AI-generated educational diagrams.
Components¶
Claims¶
- Back-translation evaluation outperforms LLM-as-a-Judge (code+image) in agreement with human raters across four models [+M]
- Back-translation yields higher human agreement on spatial checks, while LLM-as-a-Judge considerably outperforms it on the angle-label check [~W]
Related Products and Programmes¶
Key Sources¶
- Vishal Kumar, Shubhra Mishra, Rebecca Hao, Rizwaan Malik, David Broman, Dorottya Demszky. (2025). DiagramIR: An Automatic Pipeline for Educational Math Diagram Evaluation. NeurIPS 2025 Workshop: MATH-AI. https://arxiv.org/abs/2511.08283
Page metadata
| Field | Value |
|---|---|
type |
product |
id |
398-item-evaluation-dataset-of-teacher-requested-geometric-diagrams-from-coteach |
title |
398-item evaluation dataset of teacher-requested geometric diagrams from Coteach |
description |
A dataset released by Vishal Kumar et al. |
status |
review |
product_kind |
dataset |
generated |
by process:settle-candidates on 2026-10-10 |
trust |
unverified — no one has confirmed this page's content |
Frontmatter is the page's machine-readable record — see the Schema & Guide.