Direct GenAI-supported feedback improves immediate argument-quality gain over peer feedback, while reflective and hybrid designs outperform direct GenAI feedback¶

Claim · All claims
Evidence · 1 study · causal r3 · q4 · i1 small
Currency · newest study 2026 · checked against its sources 2026-10-16 (automated, 1 of 1 entry) · not reviewed by a person

Subclaims¶

q4 i1 In a cluster-randomized multisite experiment, direct GenAI-supported feedback outperformed peer feedback on four-dimension argument-quality gain (Cohen's d = 0.25), and both reflective (d = 0.29) and hybrid (d = 0.40) feedback outperformed direct GenAI feedback. → Ates 2026

Evidence¶

Ates 2026¶

Ates, H. (2026). Human-centered GenAI feedback design in higher education: a multisite experiment on direct, reflective, and hybrid approaches to scientific argumentation. International Journal of Educational Technology in Higher Education. https://doi.org/10.1186/s41239-026-00614-9

q4 · i1 · causal · r3

Multisite cluster-randomized longitudinal field experiment with 1,176 first-year students in 48 course sections across four universities; mixed-effects model, F(3, 43.8) = 18.76, p < .001. Table 3 prints small Cohen's d effect sizes: d = 0.25 for Direct GenAI vs Peer, d = 0.29 for Reflective vs Direct GenAI, and d = 0.40 for Hybrid vs Direct GenAI.

"direct GenAI-supported feedback outperformed peer feedback, and both reflective and hybrid feedback outperformed direct GenAI-supported feedback. The hybrid condition yielded the highest adjusted mean, but the hybrid–reflective contrast was not statistically significant."

Studies bearing on this claim¶

Does AI feedback or AI support improve learners' arguments and justifications, and in which designs? This table gathers the studies behind this claim and the 12 narrower claims that bear on it: 9 distinct studies (4 causal, 2 theoretical, 2 qualitative, 1 design), q1–q4, 2 of 9 with an effect size coded. One row per study, codes as each entry records them; no effect is averaged and no verdict is drawn. Built by scripts/synthesize_claim.py from eval/synthesis/; a row that looks wrong is fixed on the claim it comes from.

Discussion¶

Related Claims¶

• Reflection improves correspondence between perceived and behavioral reliance on ChatGPT — the general claim this finding bears on, with the studies behind it

• Feedback improves learning by a medium amount on average, but its effect varies widely with the information it carries, and a substantial share of feedback effects are negative — the general claim this finding bears on, with the studies behind it

• Reflective and hybrid feedback outperform direct GenAI feedback on delayed AI-free transfer — related
• Hybrid feedback outperforms direct GenAI feedback on conceptual learning; the reflective–direct contrast is positive but non-significant — related
• Randomization produced baseline-equivalent conditions across the four feedback designs — related
• Argument quality gains favored the PS-GenAI-PF group over the PF group, while elaboration gains favored the GenAI-PF group — related
• The PS-GenAI-PF group outperformed the PF group on rebuttal data and warrant and on addressing the opposing view — related
• Most individual argument structure and quality dimensions showed no significant differences across the three peer feedback conditions — related
• GenAI-supported peer feedback groups achieved higher structural complexity of argumentation artifacts than peer-feedback-only groups — related
• GenAI-supported peer feedback groups produced more suggestions and explanations and less positive affective feedback than the PF group — related
• GenAI-assisted education produces significantly higher academic achievement than non-GenAI-assisted approaches (g = 0.40) — related
• Pre-service teachers who received NLP-based adaptive feedback were better able to justify their diagnoses than those who received static feedback (reported from Sailer et al.) — related
• Students bound their reliance on AI feedback and advocate a hybrid model assigning AI broad language concerns and instructors individualized relational guidance — related