In a small case study, students given ChatGPT feedback valued its immediacy and alternative analytical perspectives, but one distrusted its answers as 'often made up' for complex data analysis¶

Claim · All claims
Evidence · 1 study (2 entries) · qualitative r2 · q2
Currency · newest study 2025 · checked against its sources 2026-10-15 (automated, 2 of 2 entries) · not reviewed by a person

Subclaims¶

q2 i? In the ChatGPT feedback group, one student (Mary) valued its immediacy and another (John) said its alternative interpretations of data broadened his thinking in statistical analysis. → Xie 2025 q2 i? One student (Tina) said ChatGPT's answers were "often made up" and that she did not fully trust it for complex data analysis; the authors file this under a theme of hallucination hindering deep learning and undermining trust. It is her perception, not a measured effect. → Xie 2025 (2)

Evidence¶

Xie 2025¶

Xie, X., Zhang, L. J., & Wilson, A. J. (2025). Comparing ChatGPT Feedback and Peer Feedback in Shaping Students' Evaluative Judgement of Statistical Analysis: A Case Study. Behavioral Sciences, 15(7), 884. https://doi.org/10.3390/bs15070884

q2 · i? · qualitative · r2

Findings section on soft evaluative judgement in the ChatGPT group; the article reports ChatGPT "expanded the scope of his thinking" by offering alternative data interpretations.

"For John, ChatGPT expanded the scope of his thinking by offering alternative ways of interpreting data—sometimes revealing patterns or perspectives he had not initially considered."

Xie 2025 (2)¶

Xie, X., Zhang, L. J., & Wilson, A. J. (2025). Comparing ChatGPT Feedback and Peer Feedback in Shaping Students' Evaluative Judgement of Statistical Analysis: A Case Study. Behavioral Sciences, 15(7), 884. https://doi.org/10.3390/bs15070884

q2 · i? · qualitative · r?

Interview response from Tina, one of three students in the ChatGPT feedback group, which the authors code under the theme "hallucination hindering deep learning and undermining trust"; she reports answers are "often made up". The study reports her perception; it does not check whether the answers were wrong or measure an effect on learning.

"The answers given by ChatGPT are often made up and not based on anything solid, so I don't fully trust it to handle complex data analysis"

Discussion¶

Related Claims¶

• Reviewed literature reports risks of overreliance, AI errors in complex tasks, and students' uncritical acceptance of AI-generated outputs — a broader claim this one bears on
• GenAI's immediate answers reduce engineering students' teacher-student interactions for generalizable tasks, while teachers remain preferred for complex problem-solving — related
• Gen-AI hallucinates sources and values and produces convincing but incorrect explanations, risking student misinformation — related
• LLM-assisted inductive qualitative coding carries risks of superficial themes, broad or redundant codes, and hallucinated interpretations, so LLMs should augment rather than replace human researchers — a broader claim this one bears on
• First-year EAP students prefer a hybrid feedback model combining AI-generated comments with instructor feedback over either source alone — related
• Thematic analysis of open responses surfaced four themes: immediacy as core value, calibrated trust, pinpointing the mistake, and progress and control — related
• The platform's out-of-scope audit protocol is designed to neutralize student distrust of AI error by rewarding verified hallucination reports — related