ChatGPT feedback required doctoral students to validate the procedural accuracy of algorithm-generated outputs, with judgement shaped by prompt quality¶

Claim · All claims
Evidence · 1 study · qualitative r? · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person · recheck by 2028: tool and model accuracy changes quickly

Subclaims¶

q2 i? Students receiving ChatGPT feedback developed hard evaluative judgement by validating procedural accuracy and interpretive soundness of AI outputs, and recognised that prompt quality influenced responses. → Xie 2025

Evidence¶

Xie 2025¶

Xie, X., Zhang, L. J., & Wilson, A. J. (2025). Comparing ChatGPT Feedback and Peer Feedback in Shaping Students' Evaluative Judgement of Statistical Analysis: A Case Study. Behavioral Sciences, 15(7), 884. https://doi.org/10.3390/bs15070884

q2 · i? · qualitative · r?

Findings on the ChatGPT feedback group in the case study. Students such as John scrutinised AI outputs against their own results, and Tina reported that a wrong prompt could produce an answer "based on a misunderstanding from the very beginning".

"ChatGPT feedback required students to validate the procedural accuracy and interpretive soundness of algorithm-generated outputs, especially in light of prompt limitations."

Discussion¶

Related Claims¶

• Both ChatGPT feedback and peer feedback prompted doctoral students to actively exercise hard evaluative judgement in statistical analysis, through different challenges — a broader claim this one bears on
• Reviewed literature reports risks of overreliance, AI errors in complex tasks, and students' uncritical acceptance of AI-generated outputs — a broader claim this one bears on
• Generative AI-generated hints can be as effective as human-generated hints for mathematics performance, but ensuring accuracy requires validation procedures that discard a substantial share of generated hints — related
• Survey items on critical evaluation and validation of ChatGPT outputs (Q11–Q13) received the lowest scores in both courses despite explicit validation instructions — reports the opposite
• The most frequently cited ethical concerns about GenAI in research were hallucinated facts, fabricated references, and misleading content — related
• Transparency about AI involvement appeared to function as an instructional intervention, shifting student reflection from feedback quality toward authority, trust, and assessment legitimacy — related
• Open questions for learnity graphs include validation, legitimacy, grading avoidance, algorithmic guidance balance, and privacy — related
• Switching from reference-based to contextual evaluation prompts improved GenAI grading accuracy — related
• GenAI feedback limitations require students to exercise evaluative judgement — a broader claim this one bears on
• Students perceive LLM chatbots as useful for learning but requiring critical use, reporting hallucinations, overconfidence, and dependency risks — related
• Students report heightened autonomy in English learning through self-directed regulation, personalized feedback, and active evaluation of ChatGPT suggestions — related
• Effects of AI-generated test case feedback depend critically on feedback validity — related