Learning-by-teaching an LLM improved assignment and project performance versus historical cohorts, but not exam performance, in an undergraduate CS course¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r1 · q2
Currency · newest study 2025 · checked against its sources 2026-10-20 (automated, 2 of 2 entries) · not reviewed by a person

Subclaims¶

q2 i? Students taught with the Socrates learning-by-teaching assignments scored significantly higher on assignments (p = 0.028) and projects (p = 0.018) than historical cohorts. → Xinming Yang 2025 q2 i? Exam score increases were not statistically significant (p = 0.693), which the authors attribute to the intervention's localized dosage across four assignments. → Xinming Yang 2025 (2)

Evidence¶

Xinming Yang 2025¶

Xinming Yang, Haasil Pujara, Jun Li. (2025). Learning by Teaching: Engaging Students as Instructors of Large Language Models in Computer Science Education. Published as a conference paper at COLM 2025. https://arxiv.org/abs/2508.05979 doi:10.48550/arxiv.2508.05979

q2 · i? · causal · r1

Quasi-experimental comparison of an after cohort (N approximately 49 to 80) against historical cohorts (N = 133 - 237) in a Computer Organization and Assembly Language course, using only unchanged coursework. The article reports "a statistically significant improvement in student performance on both Assignments (p = 0.028) and Projects (p = 0.018)"; no effect size is printed.

"Applying our approach led to a statistically significant improvement in student performance on both Assignments (p = 0.028) and Projects (p = 0.018)."

Xinming Yang 2025 (2)¶

Xinming Yang, Haasil Pujara, Jun Li. (2025). Learning by Teaching: Engaging Students as Instructors of Large Language Models in Computer Science Education. Published as a conference paper at COLM 2025. https://arxiv.org/abs/2508.05979

q2 · i? · causal · r1

Same quasi-experimental cohort comparison, exam measure. The article reports the exam increases were "not statistically significant" (p = 0.693) and suggests the four-assignment dosage "had a more localized effect that did not register as strongly on broader measures." No effect size is printed.

"Conversely, the observed score increases for Exams (p = 0.693) were not statistically significant."

Discussion¶

Related Claims¶

• AI severs the correlation between artefact quality and cognitive engagement, breaking the artefact-as-proxy for learning — the general claim this finding bears on, with the studies behind it

• Students received significantly higher homework scores during GenAI-evaluation periods, but this advantage did not transfer to downstream exams — the general claim this finding bears on, with the studies behind it

• Learning By Teaching Improves Tutor Learning — the general claim this finding bears on, with the studies behind it

• A scaffolded AI Study Coach produced no assignment-performance or concept-inventory advantage over unrestricted AI use in a master's programming course pilot — related
• Students who used LLMs on assignments scored significantly lower than non-users on the first midterm (large effect size), but this gap narrowed and was non-significant on the later midterms — related
• Prompt-literacy guidance was associated with higher transcript-coded rule-following compliance than unrestricted LLM access, mainly in process-over-answer and stepwise-hint behaviors — related
• AI-assisted software engineering courses assess primarily through participation, project work, and coding assignments rather than exams — related