LLM-generated lessons produced inauthentic references even when combined with RAG over verified academic papers¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person · recheck by 2028: tool and model accuracy changes quickly

Subclaims¶

q2 i? The Conclusion section consistently underperformed in generating authentic references across all segment configurations, even with the three-segment approach combined with RAG. → Jionghao Lin 2025

Evidence¶

Jionghao Lin 2025¶

Jionghao Lin, Jiarui Rao, Sandy Yiyang Zhao, Yuting Wang, Ashish Gurung, Amanda Barany, Jaclyn Ocumpaugh, Ryan S. Baker, and Kenneth R. Koedinger. (2025). Automatic Large Language Models Creation of Interactive Learning Lessons. arXiv preprint. https://arxiv.org/abs/2506.17356 doi:10.48550/arxiv.2506.17356

q2 · i? · design · r2

Consensus ratings (Table 2) show Authenticity of References scored 0/3 for all segmentations except four segments (1/3). The article states the RAG approach "does not inherently address the issue of citation authenticity."

"Even with the optimal segment configuration (i.e., three segments) combined with the RAG approach—which integrated academic papers containing verified references—the model still produced references that did not exist (see Table A3 in Digital Appendix)."

Discussion¶

Related Claims¶

• Of 30 verified hallucinated references, 13 appeared entirely fabricated and 17 were hybrid references combining a real title with fabricated authorship or metadata — related
• Of 100 human-audited claims, 39 are fully verifiable, 55 partially verifiable, and 6 not verifiable, with main responses showing stronger provenance than follow-ups — related
• Most of the article's body text was generated by ChatGPT from author prompts, and all references ChatGPT inserted were fictional — related
• Task decomposition into three segments yields the highest-rated LLM-generated tutor training lessons, while single-step generation yields the lowest — related
• Educator review of LLM-generated lesson material addressed seven criteria derived from generation records and caught a self-contradictory worked example before delivery — related
• Retrieval-augmented generation reduces but does not eliminate hallucinations in AI-assisted verification — a broader claim this one bears on
• Gen-AI hallucinates sources and values and produces convincing but incorrect explanations, risking student misinformation — related