ITS feedback availability interacted with task number so that experimental-group students increasingly outperformed controls in solving tasks over time¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r2 · q3
Currency · newest study 2020 · not yet checked against its sources · not reviewed by a person
Design limits · Tacoma 2020 (2): the intervention came bundled with another change (read from the study by two models, 2026-10-28)

Subclaims¶

q3 i? Adding the interaction between task number and ITS feedback availability significantly improved the logistic multilevel model predicting task solving (deviance change 13.92, p < .001). → Tacoma 2020 q3 i? While the control group's logit of solving decreased by 0.41 per task, the experimental group's decreased by only 0.09 per task, suggesting ITS feedback better supported persevering to solve tasks despite stricter assessment criteria. → Tacoma 2020 (2)

Evidence¶

Tacoma 2020¶

Tacoma, S., Heeren, B., Jeuring, J., & Drijvers, P. (2020). Intelligent Feedback on Hypothesis Testing. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-020-00218-y

q3 · i? · causal · r2

Logistic multilevel regression on task-level solving data (up to six observations per student) in the randomized experiment. The interaction term significantly improved model fit; no standardized effect size was printed, so impact is null.

"The addition of an interaction effect between task number and ITS feedback availability (M3) improved the model further: the deviance change was 13.92, which, with one degree of freedom for one extra estimated parameter, results in p < .001, hence a significant improvement to the model"

Tacoma 2020 (2)¶

Tacoma, S., Heeren, B., Jeuring, J., & Drijvers, P. (2020). Intelligent Feedback on Hypothesis Testing. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-020-00218-y

q3 · i? · causal · r2

From the final regression equation of the multilevel model: the positive interaction coefficient (0.32) means the experimental group's probability of solving declined much more slowly across tasks. The article reports this as descriptive model output with no standardized effect size.

"while for students in the control group the logit decreased by 0.41 per task, for students in the experimental group it only decreased by 0.41–0.32 = 0.09 per task"

Discussion¶

Related Claims¶

• Students receiving intelligent ITS feedback spent significantly more time and made significantly more steps on the hypothesis-testing tasks than controls — related
• In the 12-week quasi-experiment the experimental group's learning trajectory was significantly steeper than the control group's (group × time β = 0.52, 95% CI 0.31–0.73), with comparable baseline D-scores — related
• Effects of intelligent ITS feedback did not transfer to follow-up hypothesis-testing tasks without intelligent feedback — related