No single learner model was best across the six datasets, justifying a broad multi-model approach¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2021 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2024: tool and model accuracy changes quickly

Subclaims¶

q2 i? Across six learning technology datasets, the best-fitting model differed by dataset, showing no specific model is likely best for all circumstances. → Philip I. Pavlik 2021

Evidence¶

Philip I. Pavlik 2021¶

Philip I. Pavlik, Jr., Luke G. Eglington, and Leigh M. Harrell-Williams. (2021). Logistic Knowledge Tracing: A Constrained Framework for Learner Modeling. IEEE Transactions on Learning Technologies. https://doi.org/10.1109/TLT.2021.3128569

q2 · i? · design · r2

Cross-dataset model comparison fitting 12 models to six datasets with 100 split-half holdout validation runs per model and bootstrapped paired t-tests. The article reports "distinct differences between the best fitting models for the cloze and tone datasets", supporting a broad approach considering multiple model features and learning context.

"The distinct differences between the best fitting models for the cloze and tone datasets highlight the need for an overarching framework that allows various knowledge-tracing features to be compared."

Discussion¶

Related Claims¶

• The AFM+S model, which explicitly models non-zero tail error, fits simulated student data better than the standard AFM — a narrower finding that bears on this claim
• Memory-decay-based models fit fact-learning datasets better than models insensitive to memory decay — a narrower finding that bears on this claim
• The survey reports, citing Pavlik et al., that no single knowledge tracing model was always the best, and that a better model must consider multiple student features and the learning context. — possibly the same claim (merge candidate)
• The PPE memory model was the best-fitting model for the Andes physics dataset, with the recency feature also beneficial — a narrower finding that bears on this claim
• For Chinese tone learning, simpler models insensitive to memory decay fit as well as or better than memory-feature models — a narrower finding that bears on this claim
• Models able to weight performance by recency fit better on the Assistments and KDD datasets, without explicit memory-decay terms being necessary — a narrower finding that bears on this claim
• MS-BKT performs similarly to classic BKT on held-out data, with classic BKT better on most of six datasets but differences not very large — related
• Dataset size moderates the LR-versus-DKT comparison: Best-LR dominates in low and medium data regimes and DKT takes over in the high data regime — related