Across the student body as a whole, no clear statistical trend was confirmed between AI usage levels and code comprehension¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Despite descriptive patterns suggesting lower comprehension among heavier AI users, no clear statistical trend between AI usage levels and code comprehension was confirmed across the student body. → Hidetake Tanaka 2026

Evidence¶

Hidetake Tanaka 2026¶

Hidetake Tanaka, Hiroshi Igaki, Kazumasa Shimari, Kiyoshi Honda, and Naoki Fukuyasu. (2026). Practical Implementation Report on Introducing Spec-Driven Development Using AI Agents in Software Development PBL. https://arxiv.org/abs/2608.30572 doi:10.48550/arxiv.2608.30572

q2 · i? · design · r2

The authors' discussion of the RQ1–RQ3 observations states that "no clear statistical trend was confirmed between AI us age levels and code comprehension"; they attribute this to limited sample size, large standard error of the mean, and instructor interventions.

"However, across th e student body as a whole, no clear statistical trend was confirmed between AI us age levels and code comprehension."

Discussion¶

Related Claims¶

• Students who implemented code without AI (usage level 0) maintained consistently high code comprehension, while level-3 AI users showed very little comprehension in the Mid period — reports the opposite
• Cluster analysis of self-reported GenAI use identifies five profiles, including a strategic high-usage profile low on code generation but high on debugging and explanations — related
• Higher GenAI usage clusters report greater perceived difficulty, lower self-assessed understanding, and different trust in AI-generated code — related
• Team code comprehension dropped in lec09 when Implementation-phase AI usage peaked, and recovered in lec10 following instructor intervention — related
• In SDPBL with SDD introduced, AI usage varies greatly by team and phase, with all teams using AI in Implementation and Debugging but large team differences in Investigation and Planning — related
• Among LLM users, Low-Reliance students (use in under half of submissions) consistently outperformed High-Reliance students by roughly 7-8% across all three midterms — related
• Structural paths did not differ significantly across AI usage-frequency groups, though exploratory coefficients varied numerically — related