GPT-4o shows more consistent performance than Llama 3 70B across topics and metrics, while Llama 3 70B falls below 60% on the unit metric for three advanced topics¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? GPT-4o maintains a minimum correct response rate of 68.97% across all topics and metrics, whereas Llama 3 70B falls below 60% on the unit metric for three relatively advanced topics. → Liangliang Chen 2025

Evidence¶

Liangliang Chen 2025¶

Liangliang Chen, Zhihao Qin, Yiming Guo, Jacqueline Rohde, Ying Zhang. (2025). Benchmarking Large Language Models on Homework Assessment in Circuit Analysis. Int J Artif Intell Educ. https://doi.org/10.1007/s40593-025-00501-w

q2 · i? · design · r2

Summary analysis of the benchmark results across six circuit analysis topics and five metrics. The article reports GPT-4o's "minimum correct response rate of 68.97%" and Llama 3 70B's sub-60% unit-metric performance on advanced topics.

"GPT-4o demonstrates more consistent performance than Llama 3 70B, with a minimum correct response rate of 68.97% across all topics and metrics. In contrast, Llama 3 70B falls below 60% on the unit metric for three relatively advanced topics."

Discussion¶

Related Claims¶

• GPT-4o frequently makes characteristic errors in homework assessment, including failing to recognize unit and numeric equivalences, marking complete solutions incomplete, and mishandling alternative methods and notations — related
• GPT-4o and Llama 3 70B significantly outperform GPT-3.5 Turbo across all five homework-assessment metrics in circuit analysis — related
• Llama 3 70B outperforms GPT-4o on completeness and method metrics due to its leniency, but performs significantly worse on the unit metric — related
• Sinusoidal steady-state analysis is the most challenging topic for both GPT-4o and Llama 3 70B in homework assessment — related