High-frequency LLM users show higher minimum course outcomes and reduced score variability, though interaction level is not a dominant outcome predictor¶

Claim · All claims
Evidence · 1 study · associational r2 · q2
Currency · no dated study recorded · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Top-10% usage students had higher minimum assignment and exam scores and smaller variance than low and medium users, while group membership added only borderline-significant explanatory power. → usage-groups

Evidence¶

usage-groups¶

Hunter McNichols, Fareya Ikram, and Andrew Lan. 2026. The StudyChat Dataset: Analyzing Student Dialogues With ChatGPT in an Artificial Intelligence Course. In LAK26: 16th International Learning Analytics and Knowledge Conference (LAK 2026). https://doi.org/10.1145/3785022.3785029

q2 · i? · associational · r2

Regression and descriptive comparison of students grouped by total utterances (bottom 10%, middle 80%, top 10%). Group features gave a borderline significant improvement for spring e1 (F=4.08, p=0.05; group means 77.8%, 81.0%, 88.1%). High users' minimum average assignment score (92.53) and exam score (72.33) exceeded the other groups' minima.

"Our analysis demonstrates that although interaction level is not a dominant predictor of outcome, high engagement with the LLM is associated with reduced variability in course outcomes and higher minimum course outcomes."

Discussion¶

Related Claims¶

• Higher GenAI usage clusters report greater perceived difficulty, lower self-assessed understanding, and different trust in AI-generated code — reports the opposite
• Voluntary StatBot use was highly uneven and deadline-driven, with about 27% of students never using the chatbot — related
• Self-reported LLM usage frequency shows Holm-corrected associations with all five baseline AI perception outcomes, strongest for complex-task trust, accuracy trust, and critical-thinking risk — related
• GenAI/LLM modalities provide the strongest evidence for cognitive and communication outcomes, while chatbots show weak independent effects — related
• Acceptance scores are broadly held across learner characteristics, with frequency of use the only significant and weak predictor — related
• The article claims AI connectedness demonstrates incremental validity beyond usage frequency, anthropomorphism, and AI attachment — related
• Growing Achievers used StatBot significantly more than Declining Performers, driven mainly by initiating new inquiries — related
• Fall clusters of coding-question students trend toward higher exam scores than report-writer clusters — related
• Structural paths did not differ significantly across AI usage-frequency groups, though exploratory coefficients varied numerically — related
• Students who used LLMs on assignments scored significantly lower than non-users on the first midterm (large effect size), but this gap narrowed and was non-significant on the later midterms — related
• Student LLM usage patterns varied across data science courses: Course-BD showed the highest frequency and over-reliance with near-zero perceived learning effect — related