Students' conceptual-question and editing-request LLM usage is positively correlated with exam outcomes in the spring semester¶

Claim · All claims
Evidence · 1 study · associational r2 · q2
Currency · no dated study recorded · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? In spring-semester regressions, the broad dialogue act features Conceptual Questions and Editing Request were statistically significant positive correlates of exam 1 scores. → reg-broad-s25

Evidence¶

reg-broad-s25¶

Hunter McNichols, Fareya Ikram, and Andrew Lan. 2026. The StudyChat Dataset: Analyzing Student Dialogues With ChatGPT in an Artificial Intelligence Course. In LAK26: 16th International Learning Analytics and Knowledge Conference (LAK 2026). https://doi.org/10.1145/3785022.3785029

q2 · i? · associational · r2

Observational regression analysis of dialogue act counts from the StudyChat logs. In spring-semester broad-DA regressions, Conceptual Questions (coef 0.0133, p=0.024) and Editing Request (coef 0.0296, p=0.017) significantly predicted e1 scores, which the authors read as "a learning-focused usage of LLMs being positively correlated with assessment outcomes."

"We observe the broad feature for Conceptual Questions and Editing Request as positively correlated with e1 outcomes, indicating a learning-focused usage of LLMs being positively correlated with assessment outcomes."

Discussion¶

Related Claims¶

• In StudyChat, conceptual-understanding behaviors correlate positively with exam outcomes while task-delegating behaviors correlate negatively — related
• Contextual Questions dialogue acts are negatively correlated with assignment outcomes across both semesters — related
• Dialogue act features raise regression R² but do not significantly improve outcome prediction over a prior-performance baseline — related
• Conceptual Questions about mathematics are a strong negative indicator of assignment 6 scores — related