GPT-4o and Llama 3 70B significantly outperform GPT-3.5 Turbo across all five homework-assessment metrics in circuit analysis¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? GPT-4o and Llama 3 70B significantly outperform GPT-3.5 Turbo across all five metrics (completeness, method, final answer, arithmetic, units), with GPT-4o achieving correct response rates exceeding 80% on average. → Liangliang Chen 2025

Evidence¶

Liangliang Chen 2025¶

Liangliang Chen, Zhihao Qin, Yiming Guo, Jacqueline Rohde, Ying Zhang. (2025). Benchmarking Large Language Models on Homework Assessment in Circuit Analysis. Int J Artif Intell Educ. https://doi.org/10.1007/s40593-025-00501-w

q2 · i? · design · r2

Benchmark evaluation of three LLMs on 283 student solutions from 119 circuit analysis problems, with responses manually categorized as correct, partially correct, or incorrect per metric. The results show the two newer models "significantly outperform GPT-3.5 Turbo across all metrics."

"GPT-4o and Llama 3 70B significantly outperform GPT-3.5 Turbo across all metrics. Notably, GPT-4o achieves correct response rates exceeding 80% on average across all metrics."

Discussion¶

Related Claims¶

• GPT-4o frequently makes characteristic errors in homework assessment, including failing to recognize unit and numeric equivalences, marking complete solutions incomplete, and mishandling alternative methods and notations — related
• GPT-4o shows more consistent performance than Llama 3 70B across topics and metrics, while Llama 3 70B falls below 60% on the unit metric for three advanced topics — related
• Llama 3 70B outperforms GPT-4o on completeness and method metrics due to its leniency, but performs significantly worse on the unit metric — related
• Sinusoidal steady-state analysis is the most challenging topic for both GPT-4o and Llama 3 70B in homework assessment — related
• State-of-the-art multimodal LLMs cannot reliably interpret circuit diagrams and therefore cannot independently solve circuit analysis problems — related