Among LLM users, Low-Reliance students (use in under half of submissions) consistently outperformed High-Reliance students by roughly 7-8% across all three midterms¶

Claim · All claims
Evidence · 1 study · associational r1 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? The Low-Reliance group consistently outperformed the High-Reliance group across all three midterms with a stable gap of approximately 7-8%, and performed similarly to the No LLM group (3% gap). → Minju Park 2026

Evidence¶

Minju Park 2026¶

Minju Park, Ivan Orozco Vasquez, and Cristina Conati. (2026). Characterizing Students' LLM Usage Behaviors and Their Association with Learning in Critical Thinking Tasks. https://arxiv.org/abs/2605.04534 doi:10.48550/arxiv.2605.04534

q2 · i? · associational · r1

Descriptive comparison of midterm scores between the High-Reliance group (7 students using LLMs in 50-100% of submissions) and the Low-Reliance group (16 students, 5-33% of submissions). Because group sizes were highly imbalanced, the authors "did not conduct statistical tests" and interpreted only the descriptive trends in Figure 7.

"On average, Low-Reliance group consistently outperformed the High-Reliance group across all three midterms, with a stable performance gap of approximately 7-8% throughout the term."

Discussion¶

Related Claims¶

• Higher reliance on AI tools is correlated with lower critical-thinking performance, mediated by cognitive offloading — the general claim this finding bears on, with the studies behind it

• Students received significantly higher homework scores during GenAI-evaluation periods, but this advantage did not transfer to downstream exams — the general claim this finding bears on, with the studies behind it

• Students who used LLMs on assignments scored significantly lower than non-users on the first midterm (large effect size), but this gap narrowed and was non-significant on the later midterms — related
• Individual trajectories suggest heavy LLM use focused on support other than full answer generation can coexist with improving scores, whereas reliance on LLM-generated summaries was associated with declining scores — related
• Students whose LLM use was mostly student-driven support scored higher on Midterm 1 than LLM-driven users (about 10%), but the two groups converged on later midterms — related
• In an unrestricted university course, only a minority of students (23 of 68, 34%) reported using LLMs, averaging 12% of discussion-point and 11% of critical-summary submissions — related
• LLM usage varied across assignments and tasks, increasing for later assignments (especially discussion points), while Low-Reliance students concentrated use early and reduced it after Midterm 2 — related
• Across the student body as a whole, no clear statistical trend was confirmed between AI usage levels and code comprehension — related
• Students who implemented code without AI (usage level 0) maintained consistently high code comprehension, while level-3 AI users showed very little comprehension in the Mid period — related