Persistent challenges include feedback inaccuracy, learner overreliance, and limited transparency¶

Claim · All claims
Evidence · 1 study · review r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? Across 17 instances of challenges, technical limitations and pedagogical/learner-related issues were consistently reported. → Luo 2026

Evidence¶

Luo 2026¶

Luo, Y. (2026). Customizing AI for writing pedagogy: a systematic review of pedagogical goals, theoretical principles, and technical design. Education Tech Research Dev. https://doi.org/10.1007/s11423-026-10697-w

q2 · i? · review · r2

Systematic review synthesis of challenges across the 23 studies (RQ4, Fig. 7), identifying 17 challenge instances. Technical limitations included inconsistent accuracy, overgeneralized or unstable feedback, and substantial computational or financial costs.

"Pedagogical and learner-related challenges involved overreliance on AI feedback, reduced critical engagement, limited transparency of system operations, and uneven learner uptake of optional learning features"

Discussion¶

Related Claims¶

• Unstructured ChatGPT use in programming education is linked to overreliance, reduced persistence, unreliable outputs, and academic integrity risks — related
• Reviewed studies consistently raise concerns about AI accuracy, overreliance, academic integrity, and reduced critical engagement in Arab EL2 contexts — related
• Reviewed literature reports risks of overreliance, AI errors in complex tasks, and students' uncritical acceptance of AI-generated outputs — related
• Overreliance on AI is the most prevalent pedagogical limitation (27 studies, 50%), followed by reduced critical thinking (25 studies, 46%) — related
• Students report negative experiences with ChatGPT including inaccurate outputs, lack of emotional connection, and risk of overreliance, which constrain competence — related
• Unreliable AI-generated output is the most frequently reported technical limitation of AI-supported undergraduate EFL instruction, appearing in 22 of 54 studies (41%) — related
• Customized AI writing systems show benefits in writing quality, learning processes, assessment reliability, and feedback literacy — related
• The most critical challenge in integrating AI into instructional design is pedagogical alignment (14 of 28 studies, 50.0%) — related
• The most frequent limitation of AI in instructional design is content reliability and transparency (16 of 28 studies, 57.14%) — related
• AI use in training carries risks of uncritical acceptance and overreliance, including undetected conceptual errors and increased delegation of problem solving — related
• Transparency about AI involvement appeared to function as an instructional intervention, shifting student reflection from feedback quality toward authority, trust, and assessment legitimacy — related
• Open issues for dynamic assessment include cost-effectiveness, training, reliability and validity, and generalization — related
• Concerns about accuracy, reliability, academic integrity, and potential negative effects on cognitive and social development were consistently highlighted across many included studies. — a broader claim this one bears on
• ChatGPT-generated text shows inaccuracy, weak logical flow, lack of critical elaboration and non-originality in an author's first-hand test — a narrower finding that bears on this claim
• AI use also carries negative outcomes: technical instability, negative emotional impacts, over-reliance, and contextual contingency — related
• AWE tools such as Grammarly, Quillbot and DeepL Write significantly improve writing precision, fluency and grammatical proficiency through immediate and customized feedback — related
• Assessing AI feedback within PAIRR helped students develop AI literacy and writerly agency, with very few showing overconfidence in AI — related
• Graduate seminars with hands-on generative AI use and reflection increased students' comfort with AI and ability to critically assess AI outputs — related
• AWE feedback has limited depth and questionable accuracy for evaluating complex writing aspects — a narrower finding that bears on this claim
• Six implementation modes operationalize customized AI writing systems, with standalone evaluation most common — related