Students who implemented code without AI (usage level 0) maintained consistently high code comprehension, while level-3 AI users showed very little comprehension in the Mid period¶
Claim · All claims
Evidence · 1 study · designr2·q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? Students at AI usage level 0 maintained consistently high code comprehension levels throughout all periods, whereas students at AI level 3 implemented code with very little comprehension, especially during lec09–10. → Hidetake Tanaka 2026
Evidence¶
Hidetake Tanaka 2026¶
Hidetake Tanaka, Hiroshi Igaki, Kazumasa Shimari, Kiyoshi Honda, and Naoki Fukuyasu. (2026). Practical Implementation Report on Introducing Spec-Driven Development Using AI Agents in Software Development PBL. https://arxiv.org/abs/2608.30572 doi:10.48550/arxiv.2608.30572
q2 · i? · design · r2
Descriptive analysis (RQ3) plotting average instructor-assessed code comprehension (0–3 scale) by AI usage level during the Implementation phase across Early (lec07–08), Mid (lec09–10), and Late (lec11–12) periods, shown in Figure 6a; level 1 omitted as unreported.
"we can see that students at AI level 0, that is, students who implemented code without using AI, maintained consistently high code comprehension levels throughout all periods. On the other hand, students at AI level 3 impleme nted code with very little comprehension, especially during the Mid perio d (lec09–10)."
Discussion¶
Related Claims¶
- In SDPBL with SDD introduced, AI usage varies greatly by team and phase, with all teams using AI in Implementation and Debugging but large team differences in Investigation and Planning — related
- Team code comprehension dropped in lec09 when Implementation-phase AI usage peaked, and recovered in lec10 following instructor intervention — related
- Across the student body as a whole, no clear statistical trend was confirmed between AI usage levels and code comprehension — reports the opposite
- Higher GenAI usage clusters report greater perceived difficulty, lower self-assessed understanding, and different trust in AI-generated code — related
- Among LLM users, Low-Reliance students (use in under half of submissions) consistently outperformed High-Reliance students by roughly 7-8% across all three midterms — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
no-ai-use-high-comprehension-level3-low |
title |
Students who implemented code without AI (usage level 0) maintained consistently high code comprehension, while level-3 AI users showed very little comprehension in the Mid period |
description |
Students who implemented code without AI (usage level 0) maintained consistently high code comprehension, while level-3 AI users showed very little comprehension in the Mid period |
status |
review |
evidence_strength |
weak |
newest_evidence |
2026 |
generated |
by process:wiki-ingest on 2026-10-11 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — hidetake-tanaka-2026 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.