Memory-decay-based models fit fact-learning datasets better than models insensitive to memory decay¶

Claim · All claims
Evidence · 1 study (2 entries) · design r2 · q2
Currency · newest study 2021 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2024: tool and model accuracy changes quickly

Subclaims¶

q2 i? In the statistics cloze dataset, models with declarative-memory features (models 10-12) significantly improved fit beyond models insensitive to memory decay. → Philip I. Pavlik 2021 q2 i? In the McGraw Hill dataset, models explicitly tracking decay (models 10-12) did better, with large fit differences between the best and worst models. → Philip I. Pavlik 2021 (2)

Evidence¶

Philip I. Pavlik 2021¶

Philip I. Pavlik, Jr., Luke G. Eglington, and Leigh M. Harrell-Williams. (2021). Logistic Knowledge Tracing: A Constrained Framework for Learner Modeling. IEEE Transactions on Learning Technologies. https://doi.org/10.1109/TLT.2021.3128569

q2 · i? · design · r2

Analysis of the statistics cloze dataset (58,316 observations, 478 adult Mechanical Turk participants) with manipulated spacing and multi-day delays. The article reports that "models 10–12) significantly improved fit beyond those that were insensitive to memory decay," consistent with the recall task and spacing intervals.

"Models with features based on theories of declarative memory (models 10–12) significantly improved fit beyond those that were insensitive to memory decay."

Philip I. Pavlik 2021 (2)¶

Philip I. Pavlik, Jr., Luke G. Eglington, and Leigh M. Harrell-Williams. (2021). Logistic Knowledge Tracing: A Constrained Framework for Learner Modeling. IEEE Transactions on Learning Technologies. https://doi.org/10.1109/TLT.2021.3128569

q2 · i? · design · r2

Analysis of the McGraw Hill dataset (124,387 observations, 1047 college students learning fitness and nutrition facts), a task requiring recall more than skill application. The article reports large fit differences between "the best model (model 10)" and "the worst fitting model (model 1)".

"There were large differences in fit between the best model (model 10) and the worst fitting model (model 1), similar to the cloze dataset."

Studies bearing on this claim¶

Does a learner model that tracks memory decay since the last review predict recall better than one that ignores it, and for which kinds of material? This table gathers the studies behind this claim and the 9 narrower claims that bear on it: 3 distinct studies (1 design, 1 associational, 1 review), q2, 0 of 3 with an effect size coded. One row per study, codes as each entry records them; no effect is averaged and no verdict is drawn. Built by scripts/synthesize_claim.py from eval/synthesis/; a row that looks wrong is fixed on the claim it comes from.

Discussion¶

Related Claims¶

• For Chinese tone learning, simpler models insensitive to memory decay fit as well as or better than memory-feature models — reports the opposite
• Models able to weight performance by recency fit better on the Assistments and KDD datasets, without explicit memory-decay terms being necessary — related
• No single learner model was best across the six datasets, justifying a broad multi-model approach — a broader claim this one bears on
• The PPE memory model was the best-fitting model for the Andes physics dataset, with the recency feature also beneficial — related
• The survey reports, citing Pavlik et al., that no single knowledge tracing model was always the best, and that a better model must consider multiple student features and the learning context. — related