ChatGPT-3.5 feedback on EAP argumentative essays aligns closely with human instructor scoring and identifies surface-level issues, but is less effective on rhetorical structure and contextual nuance¶

Claim · All claims
Evidence · 1 study · associational r1 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? ChatGPT-3.5 demonstrated close alignment with human scoring and consistently identified surface-level issues in student argumentative essays. → Reza Farzi 2026 q2 i? ChatGPT-3.5 was less effective than the human instructor in addressing rhetorical structure and contextual nuance. → Reza Farzi 2026

Evidence¶

Reza Farzi 2026¶

Reza Farzi. (2026). Human–AI Collaboration in Academic Writing Instruction: Calibrated Feedback Model for EAP. OLBI Journal. https://doi.org/10.18192/olbij.v15i1.7773

q2 · i? · associational · r1

Mixed-methods comparison in which 30 first-year EAP students each submitted two argumentative essays evaluated independently by the instructor and ChatGPT-3.5 across three calibration stages. The study reports "close alignment with human scoring" and weaker handling of rhetorical structure and contextual nuance; no effect size is printed.

"The findings suggest that while ChatGPT-3.5 consistently identified surface-level issues and demonstrated close alignment with human scoring, it was less effective in addressing rhetorical structure and contextual nuance."

Discussion¶

Related Claims¶

• Chain-of-thought prompting yields higher-quality GenAI feedback on argumentative essays than Zero-shot prompting and teacher feedback — related
• Business Writing students valued AI feedback for higher-order and sentence-level comments but found it sometimes too general, while appreciating peers' contextual knowledge — related
• ChatGPT's exam scores were within 10% of human grades 70% of the time in a study of AI-based grading — related
• Feedback prompted with ELL status narrows to surface-level grammar and mechanics corrections, assuming limited language competency rather than addressing argument and reasoning — related
• Students perceived peer group feedback as contextually rich but sometimes subjective, and ChatGPT-4o feedback as more objective and well structured but occasionally lacking contextual sensitivity — related
• LLM-generated feedback is more comprehensive in volume than human feedback but is sometimes too generic, abstract, or misaligned with the assigned grade — related
• First-year EAP students prefer a hybrid feedback model combining AI-generated comments with instructor feedback over either source alone — related
• LLM grading performance relative to human graders is mixed: comparable in 12 reviewed cases, worse in 11, with the strongest alignment on short, straightforward tasks — a broader claim this one bears on
• Zero-shot GenAI feedback quality is statistically comparable to teacher feedback quality — related