Leave-one-out cross-validation over 36 students confirms significant accuracy and stability gains from eye-tracking in both self-explanation and exploration assessment¶

Claim · All claims
Evidence · 1 study (2 entries) · design r2–r3 · q2
Currency · newest study 2005 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2008: tool and model accuracy changes quickly

Subclaims¶

q2 i? With generic priors, cross-validated mean combined accuracy for implicitSE nodes was 62.1% (time only) versus 71.6% (time and gaze shifts), a difference significant at the 0.05 level (one-tailed t-test), with slightly higher stability. → Christina Merten and Cristina Conati 2005 q2 i? For exploration nodes, ANOVA showed statistical significance across mean accuracies and one-tailed t-tests showed each successive model differed significantly (57.3%, 65.3%, 71.6%). → Christina Merten and Cristina Conati 2005 (2)

Evidence¶

Christina Merten and Cristina Conati 2005¶

Christina Merten and Cristina Conati. (2005). Eye-Tracking to Model and Adapt to User Meta-cognition in Intelligent Learning Environments. https://www.cs.ubc.ca

q2 · i? · design · r3

Leave-one-out cross-validation using all 36 students from both studies, with thresholds and conditional probabilities set from the remaining students. Table 5 reports standard deviations of 8.1% versus 7.9% under generic priors.

"These values show improved performance with the addition of the eye-tracker as well as slightly higher stability. The performance difference is statistically significant at the 0.05 level (one-tailed t-test)"

Christina Merten and Cristina Conati 2005 (2)¶

Christina Merten and Cristina Conati. (2005). Eye-Tracking to Model and Adapt to User Meta-cognition in Intelligent Learning Environments. https://www.cs.ubc.ca

q2 · i? · design · r2

Cross-validation of exploration nodes across the three model versions (Table 6): combined accuracies of 57.3%, 65.3% and 71.6% with standard deviations 11.6%, 9.4% and 8.7% under generic priors.

"ANOVA analysis showed statistical significance in the differences within the set of mean accuracies and then one-tailed T-tests showed that the differences between each model are statistically significant"

Discussion¶

Related Claims¶

• Time and gaze-shift evidence are complementary predictors of self-explanation: time alone yields higher sensitivity, gaze shifts alone higher specificity, and the combination the best accuracy — related
• Adding eye-tracking evidence to ACE's student model significantly improves assessment of implicit self-explanation over a time-only model — a broader claim this one bears on
• In an initial eye-tracking study, time on action predicted self-explanation (16-second threshold) and gaze shifts were most reliable at detecting its absence — related
• More accurate assessment of self-explanation significantly improves the model's assessment of student learning through exploration — related
• Customizing knowledge-node priors from pretest scores improves model accuracy, but tailoring SE-tendency priors yields no significant improvement — related