Gen-AI hallucinates sources and values and produces convincing but incorrect explanations, risking student misinformation¶

Claim · All claims
Evidence · 1 study (2 entries) · design r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? Without source files, ChatGPT generated false information such as reviewing a nonexistent paper; even with the paper supplied it fabricated details like sample size. → Ivory 2026 q2 i? Incorrect AI answers came with explanations convincing enough to mislead students using Gen-AI as a learning or revision aid. → Ivory 2026 (2)

Evidence¶

Ivory 2026¶

Ivory, M., Finnerty, S., Dunn, K., Philpot, R., & Warmelink, L. (2026). The Integrity of Psychology Assessments in the AI Age: A Critical Examination. Journal of Academic Ethics. https://doi.org/10.1007/s10805-026-09729-0

q2 · i? · design · r2

In the paper review assessment (PSYC305), basic prompting reviewed a nonexistent paper with an unresolvable DOI; with the paper uploaded, ChatGPT fabricated the sample size, stating 378 participants whereas the paper reported 329.

"Further Gen-AI is prone to the generation of incorrect information and the hallucination of sources and references that do not exist."

Ivory 2026 (2)¶

Ivory, M., Finnerty, S., Dunn, K., Philpot, R., & Warmelink, L. (2026). The Integrity of Psychology Assessments in the AI Age: A Critical Examination. Journal of Academic Ethics. https://doi.org/10.1007/s10805-026-09729-0

q2 · i? · design · r2

Observation from the WBA transcripts (e.g. a developmental psychology question in PSYC205 WBA 6 where the correct answer was (a) but ChatGPT answered 'd. All of the above' with a plausible justification); the authors warn students without prior knowledge may become misinformed.

"when the Gen-AI provided incorrect answers, the associated explanations were often convincing, particularly on content outside the expertise of the researchers."

Discussion¶

Related Claims¶

• Poorly grounded GenAI self-study risks fabricated content and inferior learning relative to curated resources — possibly the same claim (merge candidate)
• Reviewed literature reports risks of overreliance, AI errors in complex tasks, and students' uncritical acceptance of AI-generated outputs — a broader claim this one bears on
• Phase VI AI-enabled systems face six documented challenges: explainability, hallucination risk, prompt sensitivity, computational cost, validation complexity, and limited large-scale evidence — a broader claim this one bears on
• Zero-shot LLM errors follow four recurring patterns: over-interpretation, failure to detect relevant information, hallucination, and failure to generate a response — related
• Students report negative experiences with ChatGPT including inaccurate outputs, lack of emotional connection, and risk of overreliance, which constrain competence — related
• AI-assisted scientific writing assignments risk fabricated references and altered chemical meaning — possibly the same claim (merge candidate)
• In a small case study, students given ChatGPT feedback valued its immediacy and alternative analytical perspectives, but one distrusted its answers as 'often made up' for complex data analysis — related
• LLM responses that are incomplete or incorrect prompt students to question, verify and improve the information, fostering critical thinking through cycles of validation and correction — related
• ChatGPT fabricates references: the authors caught it inventing a non-existent 'Ribeiro and Vala, 2020' article with full bibliographic details — related
• ChatGPT sustains coherent, human-like multi-topic conversation and produces convincing creative content, as shown by the authors' own prompting test — related
• Most of the article's body text was generated by ChatGPT from author prompts, and all references ChatGPT inserted were fictional — related
• Leading LLMs hallucinated legal authorities in 58%-88% of tested cases while failing to recognise incorrect legal assumptions in prompts
• AI systems can produce biased or inaccurate diagnostic outputs, requiring critical teacher evaluation before integration — related
• Teacher-provided correct answers and pre-prompting reduce erroneous LLM output and hallucinations compared with open chatbot use — related
• LLM-generated lessons produced inauthentic references even when combined with RAG over verified academic papers — related
• Large language models can produce fluent but factually inaccurate or fabricated outputs (AI hallucination), posing risks for uncritical users in science education — a broader claim this one bears on