The performance gap between erroneous and non-erroneous student images persists after images are digitally redrawn to remove noise¶

Claim · All claims
Evidence · 1 study · causal r1 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Differences in average content-description scores between erroneous and non-erroneous student images persist after standardized digital redrawing, across four representative VLMs. → Li Lucy 2026

Evidence¶

Li Lucy 2026¶

Li Lucy, Albert Zhang, Nathan Anderson, Ryan Knight, Kyle Lo. (2026). The Aftermath of DrawEduMath: Vision Language Models Underperform with Struggling Students and Misdiagnose Errors. https://arxiv.org/abs/2603.00925 doi:10.48550/arxiv.2603.00925

q2 · i? · causal · r1

Experiment in which the lead author digitally redrew one erroneous and one correct response per problem, yielding 336 images; Table 2 prints per-model score differences on original versus redrawn images, e.g. Gemini 2.5 Pro -0.089*** to -0.096***, with significance markers but no standardized effect size.

"Differences in average scores on content description QA between erroneous and non-erroneous student images persist after redrawing. ** p <0.01 , ***p<0.001."

Discussion¶

Related Claims¶

• Some VLMs perform near chance on binary judgments of student correctness, and error-assessment behavior is idiosyncratic across models — related
• Gold text descriptions improve VLM correctness-and-error assessment, but performance still lags other question types — related
• The LWPA advantage on the End-of-Unit exam persists after removing oral comprehension scores — related
• VLMs achieve higher accuracy on non-erroneous than erroneous student math responses even when controlling for the math problem — related