Unstructured ChatGPT use in programming education is linked to overreliance, reduced persistence, unreliable outputs, and academic integrity risks¶

Claim · All claims
Evidence · 1 study (2 entries) · review r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Frequent ChatGPT use showed a negative correlation with programming test performance, and heavy reliance was associated with weaker problem-solving effort and fewer effective software tests. → Grume 2026 q2 i? ChatGPT outputs in programming tasks were frequently incomplete, incorrect, or unusable, and AI-content detectors showed poor accuracy. → Grume 2026 (2)

Evidence¶

Grume 2026¶

Grume, J. C., Miranda, J. P. P., De Leon, A. P., Salenga, J. L., Hernandez, H. E., Castro, M. A., Maniago, V. G. M., Canlas, J. D., & Quiambao, J. B. (2026). Pedagogical Promise and Peril of AI: A Text Mining Analysis of ChatGPT Research Discussions in Programming Education. In M. B. Garcia (Ed.), Pedagogical Innovations in Computer Science Education (pp. 305-330). IGI Global Scientific Publishing. https://doi.org/10.4018/979-8-3373-6546-6.ch010

q2 · i? · review · r2

The chapter reports, citing Lepp and Kaimre (2025), a study of students' perceptions, reported use, and performance that "found a negative correla tion between frequent ChatGPT use and programming test performance." The chapter also cites Mezzaro et al. (2024) on heavy reliance producing fewer and less effective software tests. No correlation coefficient is printed.

"Lepp and Kaimre (2025) found a negative correla tion between frequent ChatGPT use and programming test performance."

Grume 2026 (2)¶

Grume, J. C., Miranda, J. P. P., De Leon, A. P., Salenga, J. L., Hernandez, H. E., Castro, M. A., Maniago, V. G. M., Canlas, J. D., & Quiambao, J. B. (2026). Pedagogical Promise and Peril of AI: A Text Mining Analysis of ChatGPT Research Discussions in Programming Education. In M. B. Garcia (Ed.), Pedagogical Innovations in Computer Science Education (pp. 305-330). IGI Global Scientific Publishing. https://doi.org/10.4018/979-8-3373-6546-6.ch010

q2 · i? · review · r2

The chapter reports, citing Cowan et al. (2023), that "only 30% of ChatGPT outputs were fully usable" in programming learning interactions, alongside reported hallucinated responses and repeated errors in code explanations from other reviewed studies.

"Cowan et al. (2023) found that only 30% of ChatGPT outputs were fully usable, while failed conversations occurred frequently."

Discussion¶

Related Claims¶

• A 2026 text-mining review chapter reports that studies of ChatGPT in programming education find benefits where it is used with structured pedagogical guidance and teacher facilitation — reports the opposite
• Reviewed literature reports risks of overreliance, AI errors in complex tasks, and students' uncritical acceptance of AI-generated outputs — a broader claim this one bears on
• Teachers raise big-picture concerns about GenAI including surveillance, over-reliance, reduced teacher collaboration, job replacement, and training needs — related
• Students report negative experiences with ChatGPT including inaccurate outputs, lack of emotional connection, and risk of overreliance, which constrain competence — related
• Unreliable AI-generated output is the most frequently reported technical limitation of AI-supported undergraduate EFL instruction, appearing in 22 of 54 studies (41%) — related
• Overreliance on AI is the most prevalent pedagogical limitation (27 studies, 50%), followed by reduced critical thinking (25 studies, 46%) — a broader claim this one bears on
• Reviewed studies consistently raise concerns about AI accuracy, overreliance, academic integrity, and reduced critical engagement in Arab EL2 contexts — related
• The most frequent co-occurrence pattern pairs AI system performance limitations with learner cognitive and autonomy risks, reported together in 25 studies (46%), as association rather than causation — related
• Academic integrity concerns were the third most prevalent pedagogical limitation, reported in 20 of 54 studies (37%) — related
• Automated verification of AI-drafted content produced mixed results: strong accuracy and integrity scores but only 5 of 10 chapters meeting the pedagogical-progression bar — related
• Persistent challenges include feedback inaccuracy, learner overreliance, and limited transparency — related
• Limited output accuracy and over-reliance are the most pressing challenges Chinese HSS students report in GenAI use, alongside strong ethical concern — related
• The most frequent limitation of AI in instructional design is content reliability and transparency (16 of 28 studies, 57.14%) — related
• AI use in training carries risks of uncritical acceptance and overreliance, including undetected conceptual errors and increased delegation of problem solving — related
• AI use also carries negative outcomes: technical instability, negative emotional impacts, over-reliance, and contextual contingency — related
• Viewers of learning-oriented content describe ChatGPT as a tutor or thinking partner, while viewers of output-oriented content raise over-reliance and cognitive-offloading concerns — related
• Agentivism generates six empirically testable propositions linking delegation structure, verification, reconstruction, and reduced-support transfer to learning outcomes — related
• Concerns about accuracy, reliability, academic integrity, and potential negative effects on cognitive and social development were consistently highlighted across many included studies. — a broader claim this one bears on