GPT-4's error-diagnosis accuracy varies by error type: near-perfect on correct answers, weakest on Wrong Field and multiple-error responses¶
Claim · All claims
Evidence · 1 study · designr3·q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person · recheck by 2028: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? GPT-4 recognized matching correct answers 99% of the time but failed to recognize Wrong Field errors nearly 30% of the time. → Reddig 2025
Evidence¶
Reddig 2025¶
Reddig, J. M., Arora, A., & MacLellan, C. J. (2025). Generating In-Context, Personalized Feedback for Intelligent Tutors with Large Language Models. Int J Artif Intell Educ. https://doi.org/10.1007/s40593-025-00505-6
q2 · i? · design · r3
Error-type breakdown of the same GPT-4 diagnosis analysis across all tutoring topics (Table 4, final prompt). Correct Answer accuracy was 99.0% (238), Wrong Field 72.3% (162), and multiple-error answers 71.67% (42).
"The LLM is most adept at identifying a correct answer, recognizing a matching answer 99% of the time, and questioning the validity of the correct answer only 1% of the time. GPT-4 performed the poorest on Wrong Field errors, unable to recognize nearly 30% of the time when a student had skipped a step in the problem solving process."
Discussion¶
Related Claims¶
- Given sufficient prompt context, GPT-4 describes student errors in a College Algebra ITS with 87.8% accuracy — related
- Adding tutor interface and skill-estimate context to the prompt substantially improves GPT-4's error-diagnosis accuracy — related
- GPT models showed lower accuracy with more rubric-related errors (28% vs. 13% for Gemini 3 Flash) and near parity between false positives and negatives — related
- Fine-tuned classifiers detect only about 57% of correct-answer misconceptions despite near-perfect wrong-answer classification — related
- GPT-4 generates targeted corrective feedback for 66% of student errors, but 35% of hints are too general, incorrect, or give away the answer — related
- LLM misconception analysis that ties student errors to instructional content reveals insights not apparent from performance data alone — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
gpt4-diagnosis-accuracy-varies-by-error-type |
title |
GPT-4's error-diagnosis accuracy varies by error type: near-perfect on correct answers, weakest on Wrong Field and multiple-error responses |
description |
GPT-4's error-diagnosis accuracy varies by error type: near-perfect on correct answers, weakest on Wrong Field and multiple-error responses |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2025 |
recheck_by |
2028 |
generated |
by process:wiki-ingest on 2026-10-12 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — reddig-2025 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.