In the Java-based CS1 course at TU Delft, GATs significantly improved immediate post-intervention performance pooled across topics, driven mainly by the While-loop intervention¶

Claim · All claims
Evidence · 1 study · causal r2 · q3
Currency · no dated study recorded · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? GATs significantly improved immediate performance across topics at TU Delft, mainly driven by the While-loop intervention where the treatment group outperformed the control group. → delft-immediate

Evidence¶

delft-immediate¶

Yuri Noviello, Naaz Sibia, Anastasiia Birillo, Thomas Overklift Vaupel Klein, Michael Liut, and Gosia Migut. 2026. AI-Generated Traces for Novice Programmers: Learning Effects and Learner Differences in a Multi-Institutional Study. Proceedings of the 31st ACM Conference on Innovation and Technology in Computer Science Education V. 1 (ITiCSE 2026). https://doi.org/10.1145/3803400.3809346

q3 · i? · causal · r2

Randomized experiment in a 9-week Java CS1 course at TU Delft (pooled N=151; While-loop N=138), analyzed with linear mixed models controlling for topic and prior experience; Table 1 row 21 reports Δ=5.420, p=0.003, padj=0.017, and the While-loop row reports Δ=8.227, p=0.003, padj=0.0159. Other topics showed positive but non-significant differences that failed correction.

"Study 1 at TU Delft:GATssignificantly improved immediate per- formance across topics (Table 1, row 21), mainly driven by the While-loopintervention (Table 1, row 1), where the treatment group outperformed the control group."

Discussion¶

Related Claims¶

• Engagement profiles significantly moderated GATs' immediate performance effects: low- and high-engagement profiles benefited while the mid-engagement profile showed a small decrement — related
• GATs produced no reliable immediate learning-experience effects on frustration, cognitive load, or situational interest at either institution — related
• In the Python-based CS1 course at UofT, GATs produced no significant overall or topic-specific immediate performance effects — related
• Across both institutions, GATs showed no significant effect on long-term summative exam performance — related
• Instruction type predicts post-test performance with a large effect size after controls (linear and quantile regression) — related
• At UofT, GATs produced a reliable end-of-course increase in Constructive engagement, with no effects on Passive, Active engagement, or MSLQ elaboration — related