Skip to content

NAEP 2023 Automatic Math Scoring Challenge dataset and competition

Product or Programme · All products and programmes
Evidence · 1 claim (1 for) · 1 study (1 design), q2 · 0 of 1 report an effect size · 1 claim rests on one study

Description

A National Center for Education Statistics challenge dataset and benchmark comprising scored and withheld-score grade 4 and grade 8 NAEP mathematics constructed responses for evaluating automated scoring accuracy and demographic fairness.

Components

  • NAEP 2023 Automated Math Scoring Challenge dataset and competition: The NAEP 2023 Automatic Math Scoring Challenge provided a training dataset of 251,370 responses from 10 items (five grade 4, five grade 8) and a test dataset of 27,930 responses with withheld scores. Items required short or extended constructed responses with maximum scores between 2 and 5. The competition goal was to predict scores within QWK 0.05 of human inter-rater reliability without demographic bias (pairwise SMD > 0.10). (Morris et al. (2025))

Claims

Key Sources

  • Morris, W., Holmes, L., Choi, J. S., & Crossley, S. (2025). Automated Scoring of Constructed Response Items in Math Assessment Using Large Language Models. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-024-00418-w
Page metadata
Field Value
type product
id naep-2023-automatic-math-scoring-challenge-dataset-and-competition
title NAEP 2023 Automatic Math Scoring Challenge dataset and competition
description A National Center for Education Statistics challenge dataset and benchmark comprising scored and withheld-score grade 4 and grade 8 NAEP mathematics constructed responses for evaluating automated scoring accuracy and demographic fairness.
status review
product_kind dataset
generated by process:settle-candidates on 2026-10-28
trust unverified — no one has confirmed this page's content

Frontmatter is the page's machine-readable record — see the Schema & Guide.