A downward trend in final grades is observed with increased average LLM use across all task types¶

Claim · All claims
Evidence · 1 study · causal r1 · q2
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Scatterplots show decreasing final grades with increased average LLM use across all types of LLM usage. → Jošt 2024

Evidence¶

Jošt 2024¶

Jošt, G.; Taneski, V.; Karakatič, S. (2024). The Impact of Large Language Models on Programming Education and Student Learning Outcomes. Appl. Sci. 14, 4115. https://doi.org/10.3390/app14104115

q2 · i? · causal · r1

Visual inspection of jittered scatterplots (Figure 3) relating reported LLM use to final grades in the thirty-two-participant experiment; the authors describe this as a visible trend, with formal significance tested separately via Spearman correlation.

"Looking at the overall use of LLMs, there is a visible trend of decreasing final grades with increased average LLM use."

Discussion¶

Related Claims¶

• LLM use for code generation is significantly negatively correlated with student final grades
• LLM use for debugging is significantly negatively correlated with student final grades
• LLM use for seeking additional explanations does not significantly impact final grades — related
• Higher average LLM use across all tasks is significantly negatively correlated with final grades in undergraduate programming education — related