GPT-4o frequently makes characteristic errors in homework assessment, including failing to recognize unit and numeric equivalences, marking complete solutions incomplete, and mishandling alternative methods and notations¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2025 · checked against its sources 2026-10-27 (automated, 1 of 1 entry) · not reviewed by a person

Subclaims¶

q2 i? GPT-4o's recurring assessment errors include unit-conversion equivalence failures, number-format equivalence failures, false incompleteness judgments, rejection of correct alternative methods, and failures to recognize exchanged variable notations. → Liangliang Chen 2025

Evidence¶

Liangliang Chen 2025¶

Liangliang Chen, Zhihao Qin, Yiming Guo, Jacqueline Rohde, Ying Zhang. (2025). Benchmarking Large Language Models on Homework Assessment in Circuit Analysis. Int J Artif Intell Educ. https://doi.org/10.1007/s40593-025-00501-w

q2 · i? · design · r2

Qualitative error taxonomy drawn from the authors' review of GPT-4o responses (Table 10), with example types such as unit conversion, false incompleteness, number format, alternative method, and different notations, each illustrated with a worked example and note.

"Although GPT-4o shows the most promising evaluation results according to Table3 and the analyses above, it is important to summarize the errors it frequently makes. Table10 lists the types of errors GPT-4o tends to make along with associated examples."

Discussion¶

Related Claims¶

• Llama 3 70B outperforms GPT-4o on completeness and method metrics due to its leniency, but performs significantly worse on the unit metric — related
• GPT-4o and Llama 3 70B significantly outperform GPT-3.5 Turbo across all five homework-assessment metrics in circuit analysis — related
• Zero-shot LLM errors follow four recurring patterns: over-interpretation, failure to detect relevant information, hallucination, and failure to generate a response — related
• GPT-4o's correctness-labeling errors concentrate on final turns requiring numerical calculation, and its main KC-labeling error is assigning too few standards to a turn. — related
• State-of-the-art multimodal LLMs cannot reliably interpret circuit diagrams and therefore cannot independently solve circuit analysis problems — related
• Recurring risks of GenAI in STEAM education include hallucination, bias, superficial completion strategies, and compromised assessment validity — related
• GPT-4o shows more consistent performance than Llama 3 70B across topics and metrics, while Llama 3 70B falls below 60% on the unit metric for three advanced topics — related
• Qualitative error analysis yields five recurring error types, with over-inference most prevalent (33.5%), followed by evidence misidentification (23.4%) and hallucination (19.8%) — related