GenAI feedback limitations require students to exercise evaluative judgement¶

Claim · All claims
Evidence · 1 study · theoretical r3 · q2
Currency · newest study 2025 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person

Subclaims¶

q2 i? GenAI may produce inaccurate, biased, vague or fabricated responses, so students cannot place great trust in its outputs and must exercise evaluative judgement. → Ying Zhan 2025

Evidence¶

Ying Zhan 2025¶

Ying Zhan, David Boud, Phillip Dawson & Zi Yan (2025) Generative artificial intelligence as an enabler of student feedback engagement: a framework, Higher Education Research & Development, 44:5, 1289-1304, https://doi.org/10.1080/07294360.2025.2476513

q2 · i? · theoretical · r3

Literature synthesis in the processing-feedback section. The paper cites Essien et al.'s identification of the 'hallucination effect' and argues that 'students cannot place great trust in the outputs of GenAI, and they need to exercise evaluative judgement skills in assessing GenAI outputs and processes'.

"They identified the 'hallucination effect', where ChatGPT sometimes generated responses that were wrong or not suitable for the context."

Discussion¶

Learner Variables¶

• Metacognition — outcome: instruction changes it

Related Claims¶

• Students report negative experiences with ChatGPT including inaccurate outputs, lack of emotional connection, and risk of overreliance, which constrain competence — related
• Human guidance remains essential for ethical grounding when integrating AI with situated learning, including resisting complacency toward generative AI outputs — related
• The most frequent limitation of AI in instructional design is content reliability and transparency (16 of 28 studies, 57.14%) — a broader claim this one bears on
• Reviewed literature reports risks of overreliance, AI errors in complex tasks, and students' uncritical acceptance of AI-generated outputs — related
• Students perceive LLM chatbots as useful for learning but requiring critical use, reporting hallucinations, overconfidence, and dependency risks — possibly the same claim (merge candidate)
• The most frequently cited ethical concerns about GenAI in research were hallucinated facts, fabricated references, and misleading content — related
• The paper argues authenticity in AI-mediated assessment is achieved through design rather than detection — related
• Both ChatGPT feedback and peer feedback prompted doctoral students to actively exercise hard evaluative judgement in statistical analysis, through different challenges — a narrower finding that bears on this claim
• High-level feedback on reasoning and synthesis should remain a human responsibility for the foreseeable future — related
• Large language models can produce fluent but factually inaccurate or fabricated outputs (AI hallucination), posing risks for uncritical users in science education
• ChatGPT-generated text shows inaccuracy, weak logical flow, lack of critical elaboration and non-originality in an author's first-hand test — a narrower finding that bears on this claim
• Generative AI platforms cannot yet independently lead meaningful innovation; at best they augment human intelligence for innovation — related
• AI systems can produce biased or inaccurate diagnostic outputs, requiring critical teacher evaluation before integration — a narrower finding that bears on this claim
• AI literacy requires discipline-specific calibration beyond prompt-writing skills — related
• ChatGPT feedback required doctoral students to validate the procedural accuracy of algorithm-generated outputs, with judgement shaped by prompt quality — a narrower finding that bears on this claim
• LLM output can be misleading or erroneous because a stochastic element in text generation sometimes selects less likely continuations, so it requires continuous supervision and critical evaluation by domain experts. — a narrower finding that bears on this claim
• Assessing AI feedback within PAIRR helped students develop AI literacy and writerly agency, with very few showing overconfidence in AI — related
• Judgement is theorized as the evaluative locus of educational integrity within GenAI-augmented authorship — related
• Most students reported improved output-evaluation competence after comparing their own text analysis with LLM-generated analyses — a narrower finding that bears on this claim