Feedback quality did not differ by students' language status for either human evaluators or ChatGPT¶
Claim · All claims
Evidence · 1 study · causalr2·q3
Currency · newest study 2023 · not yet checked against its sources · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q3 i? Feedback given to EO/IFEP students was not statistically different from feedback given to EL and RFEP students, for both AI and human raters (Scheffe-corrected ANOVAs). → Steiss 2023
Evidence¶
Steiss 2023¶
Steiss, Jacob; Tate, Tamara; Graham, Steve; Cruz, Jazmin; Hebert, Michael; Wang, Jiali; Moon, Youngsun; Tseng, Waverly; Warschauer, Mark. (2023). Comparing the Quality of Human and ChatGPT Feedback on Students' Writing. https://eric.ed.gov/?q=Comparing+the+Quality+of+Human+and+ChatGPT+Feedback+on+Students+Writing
q3 · i? · causal · r2
Scheffe-corrected ANOVAs on coded feedback ratings for the blocked sample (50 ELs, 50 RFEP, 100 fluent speakers) found no statistically significant language-status differences "with respect to both the AI and the human raters."
"Feedback given to students who were either initially fluent or English-only speakers (collectively, EO/IFEP) was not statistically different compared to that given to students classified as English learners (ELs) and reclassified fluent speakers (RFEP) (using Scheffe correction for multiple comparisons)."
Discussion¶
Learner Variables¶
- English-Learner Status — moderator: an instructional effect differs with it
Related Claims¶
- Differences between human and ChatGPT feedback quality were small, with means within one point and partial eta-squared ranging from .01 to .12 — related
- Zero-shot GenAI feedback quality is statistically comparable to teacher feedback quality — related
- Feedback prompted with ELL status narrows to surface-level grammar and mechanics corrections, assuming limited language competency rather than addressing argument and reasoning — related
- VIP effects did not differ between English language learners and fluent English speakers in the study authors' analyses — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
feedback-quality-no-language-status-difference |
title |
Feedback quality did not differ by students' language status for either human evaluators or ChatGPT |
description |
Feedback quality did not differ by students' language status for either human evaluators or ChatGPT |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2023 |
generated |
by process:wiki-ingest on 2026-10-15 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — steiss-2023 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.