LaTeX-format dataset of 283 real student solutions from 119 circuit analysis problems¶
Product or Programme · All products and programmes
Evidence · 5 claims (5 for) · 1 study (1 design),q2· 0 of 1 report an effect size · 5 claims rest on one study
Description¶
A Georgia Tech dataset of manually proofread LaTeX-converted student and reference solutions for undergraduate circuit-analysis homework, created by Liangliang Chen et al. to evaluate automated assessment of solution completeness, methods, answers, arithmetic, and units.
Components¶
Claims¶
- GPT-4o frequently makes characteristic errors in homework assessment, including failing to recognize unit and numeric equivalences, marking complete solutions incomplete, and mishandling alternative methods and notations [+W]
- GPT-4o shows more consistent performance than Llama 3 70B across topics and metrics, while Llama 3 70B falls below 60% on the unit metric for three advanced topics [+W]
- Llama 3 70B outperforms GPT-4o on completeness and method metrics due to its leniency, but performs significantly worse on the unit metric [+W]
- Sinusoidal steady-state analysis is the most challenging topic for both GPT-4o and Llama 3 70B in homework assessment [+W]
- GPT-4o and Llama 3 70B significantly outperform GPT-3.5 Turbo across all five homework-assessment metrics in circuit analysis [+W]
Related Products and Programmes¶
-
Key Sources¶
- Liangliang Chen, Zhihao Qin, Yiming Guo, Jacqueline Rohde, Ying Zhang. (2025). Benchmarking Large Language Models on Homework Assessment in Circuit Analysis. Int J Artif Intell Educ. https://doi.org/10.1007/s40593-025-00501-w
Page metadata
| Field | Value |
|---|---|
type |
product |
id |
latex-format-dataset-of-283-real-student-solutions-from-119-circuit-analysis-problems |
title |
LaTeX-format dataset of 283 real student solutions from 119 circuit analysis problems |
description |
A Georgia Tech dataset of manually proofread LaTeX-converted student and reference solutions for undergraduate circuit-analysis homework, created by Liangliang Chen et al. |
status |
review |
product_kind |
dataset |
generated |
by process:settle-candidates on 2026-10-27 |
trust |
unverified — no one has confirmed this page's content |
Frontmatter is the page's machine-readable record — see the Schema & Guide.