VLMs achieve higher accuracy on non-erroneous than erroneous student math responses even when controlling for the math problem¶

Claim · All claims
Evidence · 1 study · associational r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? Across 11 VLMs evaluated on DrawEduMath, non-erroneous student responses significantly correspond with higher content-description accuracy after problem fixed effects are applied. → Li Lucy 2026

Evidence¶

Li Lucy 2026¶

Li Lucy, Albert Zhang, Nathan Anderson, Ryan Knight, Kyle Lo. (2026). The Aftermath of DrawEduMath: Vision Language Models Underperform with Struggling Students and Misdiagnose Errors. https://arxiv.org/abs/2603.00925 doi:10.48550/arxiv.2603.00925

q2 · i? · associational · r2

Ordinary least squares regression with problem fixed effects over DrawEduMath content description QA for 11 VLMs, contrasting erroneous and non-erroneous student responses; the article prints per-model β1 estimates, and reports "all p<1.0 −12" in Table 1, with no standardized effect size.

"These values show that even after controlling for prob- lem, non-erroneous student responses significantly correspond with higher VLM performance."

Discussion¶

Related Claims¶

• Some VLMs perform near chance on binary judgments of student correctness, and error-assessment behavior is idiosyncratic across models — related
• Gold text descriptions improve VLM correctness-and-error assessment, but performance still lags other question types — related
• The performance gap between erroneous and non-erroneous student images persists after images are digitally redrawn to remove noise — related
• Vision-assisted extraction identified chart type in all sampled images but showed ±5–10% numeric estimation error and two failure modes correctable via mandatory review — related