The most frequent limitation of AI in instructional design is content reliability and transparency (16 of 28 studies, 57.14%)¶

Claim · All claims
Evidence · 1 study · review r2 · q3
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q3 i? Content Reliability and Transparency (n = 16) was the most frequent limitation, including inaccurate, fabricated, or contradictory outputs; Technical/Tool-Specific Limits (n = 12) and Contextual and Cultural Blindness (n = 10) followed. → Pınar Nuhoğlu Kibar & Hale Ilgaz 2026

Evidence¶

Pınar Nuhoğlu Kibar & Hale Ilgaz 2026¶

Pınar Nuhoğlu Kibar & Hale Ilgaz. (2026). The intersection of artificial intelligence and instructional design practice: a systematic review. Educational Technology Research and Development. https://doi.org/10.1007/s11423-026-10624-z

q3 · i? · review · r2

Thematic synthesis of limitations (Table 7). The review notes Parsons & Curry (2024) characterize GenAI as a "black box," with limited visibility into decision-making constraining interpretability, and that LLM performance "remains uneven across content areas" (Hu et al., 2024).

"Practitioners most frequently encountered limitations (Table 7) related to Content Reliability and Transparency (n = 16), which included inaccurate, fabricated, or contradictory information in AI outputs, such as outdated knowledge, unverifiable references, and overly technical or incomplete explanations."

Discussion¶

Related Claims¶

• Students report negative experiences with ChatGPT including inaccurate outputs, lack of emotional connection, and risk of overreliance, which constrain competence — related
• Persistent challenges include feedback inaccuracy, learner overreliance, and limited transparency — related
• The most frequently cited ethical concerns about GenAI in research were hallucinated facts, fabricated references, and misleading content — related
• Unreliable AI-generated output is the most frequently reported technical limitation of AI-supported undergraduate EFL instruction, appearing in 22 of 54 studies (41%) — related
• Reviewed literature reports risks of overreliance, AI errors in complex tasks, and students' uncritical acceptance of AI-generated outputs — related
• Poorly grounded GenAI self-study risks fabricated content and inferior learning relative to curated resources — related
• Unstructured ChatGPT use in programming education is linked to overreliance, reduced persistence, unreliable outputs, and academic integrity risks — related
• Limited contextual understanding of pragmatic, cultural, and discourse information is a major technical limitation, reported in 16 of 54 studies (30%) — related
• Favorable assessments of LLM review workflows frequently coexist with reported unmet reliability or performance requirements — related
• Most participants articulated technical and contextual limitations of AI evaluation, including handwriting misreading and unawareness of the instructor's rating system — a narrower finding that bears on this claim
• Transparency about AI involvement appeared to function as an instructional intervention, shifting student reflection from feedback quality toward authority, trust, and assessment legitimacy — related
• GenAI feedback limitations require students to exercise evaluative judgement — a narrower finding that bears on this claim
• General-purpose LLMs show accuracy and reliability limits, including hallucinations and task-specific accuracy below perfect, making them risky as expert advice sources — a narrower finding that bears on this claim
• The most common purpose of AI in instructional design is enhancing or assisting the design process itself (22 of 28 studies, 78.5%) — related
• Customized AI writing systems show benefits in writing quality, learning processes, assessment reliability, and feedback literacy — related
• Online reading for academic purposes is a complex activity involving analysing task requirements, assessing information needs, accessing relevant contents, and evaluating relevance and reliability of information. — related
• Reported limitation profiles differ by automation method, with prompt sensitivity especially frequent in LLM papers — related
• Large language models can produce fluent but factually inaccurate or fabricated outputs (AI hallucination), posing risks for uncritical users in science education — a narrower finding that bears on this claim