Dialogue act features raise regression R² but do not significantly improve outcome prediction over a prior-performance baseline¶
Claim · All claims
Evidence · 1 study · associationalr2·q2
Currency · no dated study recorded · not yet checked against its sources · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? Adding broad and specific dialogue act count features increased mean R² substantially, but F-tests did not indicate statistically significant improvements for most assessments. → reg-null
Evidence¶
reg-null¶
Hunter McNichols, Fareya Ikram, and Andrew Lan. 2026. The StudyChat Dataset: Analyzing Student Dialogues With ChatGPT in an Artificial Intelligence Course. In LAK26: 16th International Learning Analytics and Knowledge Conference (LAK 2026). https://doi.org/10.1145/3785022.3785029
q2 · i? · associational · r2
Regression series predicting normalized assessment scores from a prior-score baseline, total utterance counts, and 8 broad or 31 specific DA counts across 9 assessments in both semesters. Specific-DA models raised mean R² to 0.4868 (fall) and 0.4012 (spring), but the authors report the models are likely overfitting or statistically underpowered by a limited sample size.
"F-Tests conducted for each regression do not indicate results that are statistically significant, with the highest 𝑅2 improvement of0.19and resulting F-Score of 𝐹= 1.81,𝑝= 0.09."
Discussion¶
Related Claims¶
- Students' conceptual-question and editing-request LLM usage is positively correlated with exam outcomes in the spring semester — related
- PTA traits show stronger statistical explanatory power than dialogue-act and Bloom's taxonomy coded baselines on StudyChat — related
- Ablation of feature-vector models: time-window features add no predictive power to logistic regression but boost a feedforward network, and total count features substantially boost performance on all datasets — related
- Conceptual Questions about mathematics are a strong negative indicator of assignment 6 scores — related
- Scaffolding level provides no consistent additional predictive information about subsequent exam performance beyond prior achievement and observed dialogue behavior — related
- PTA-derived traits significantly improve explained exam-score variance over a prior-outcome baseline in one semester of StudyChat but not the other — related
- Contextual Questions dialogue acts are negatively correlated with assignment outcomes across both semesters — a narrower finding that bears on this claim
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
da-features-not-significant-over-baseline |
title |
Dialogue act features raise regression R² but do not significantly improve outcome prediction over a prior-performance baseline |
description |
Dialogue act features raise regression R² but do not significantly improve outcome prediction over a prior-performance baseline |
status |
review |
evidence_strength |
weak |
generated |
by process:wiki-ingest on 2026-10-11 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — reg-null |
Frontmatter is the page's machine-readable record — see the Schema & Guide.