LLM use for seeking additional explanations does not significantly impact final grades¶

Claim · All claims
Evidence · 1 study · causal r1 · q3
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? Use of LLMs for seeking additional explanations shows no statistically significant correlation with final grades (Spearman's rho = -0.201, p = 0.135). → Jošt 2024

Evidence¶

Jošt 2024¶

Jošt, G.; Taneski, V.; Karakatič, S. (2024). The Impact of Large Language Models on Programming Education and Student Learning Outcomes. Appl. Sci. 14, 4115. https://doi.org/10.3390/app14104115

q3 · i? · causal · r1

Spearman correlation from the same experiment; the bootstrap confidence interval (−0.523, 0.220) encompasses zero, reflecting non-significance. The article prints "Spearman's rho = −0.201, p = 0.135".

"H2b's exploration of LLMs for seeking additional explanations did not yield a statistically significant impact on grades (Spearman's rho = −0.201, p = 0.135)"

Discussion¶

Related Claims¶

• Higher average LLM use across all tasks is significantly negatively correlated with final grades in undergraduate programming education — a broader claim this one bears on
• LLM use for code generation is significantly negatively correlated with student final grades — related
• LLM use for debugging is significantly negatively correlated with student final grades — related
• By the final survey wave, using LLMs for debugging and understanding concepts was nearly universal among surveyed students
• A downward trend in final grades is observed with increased average LLM use across all task types — related