LLM use for code generation is significantly negatively correlated with student final grades¶
Claim · All claims
Evidence · 1 study · causalr1·q3
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q3 i? Use of LLMs for generating code correlates negatively with final grades (Spearman's rho = -0.305, p = 0.045). → Jošt 2024
Evidence¶
Jošt 2024¶
Jošt, G.; Taneski, V.; Karakatič, S. (2024). The Impact of Large Language Models on Programming Education and Student Learning Outcomes. Appl. Sci. 14, 4115. https://doi.org/10.3390/app14104115
q3 · i? · causal · r1
One-sided Spearman correlation with bootstrap confidence intervals from the same thirty-two-participant experiment; the confidence interval (−0.595, −0.058) excludes zero. The article prints "Spearman's rho = −0.305, p = 0.045".
"the analysis revealed a significant inverse relationship between the use of LLMs for generating code and final grades (Spearman's rho = −0.305, p = 0.045)"
Discussion¶
Related Claims¶
- Higher average LLM use across all tasks is significantly negatively correlated with final grades in undergraduate programming education — a broader claim this one bears on
- LLM use for debugging is significantly negatively correlated with student final grades — related
- LLM use for seeking additional explanations does not significantly impact final grades — related
- A downward trend in final grades is observed with increased average LLM use across all task types
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
llm-code-generation-negative-correlation-grades |
title |
LLM use for code generation is significantly negatively correlated with student final grades |
description |
LLM use for code generation is significantly negatively correlated with student final grades |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2024 |
generated |
by process:wiki-ingest on 2026-10-12 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — jošt-2024 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.