Individualized Knowledge Tracing parameters fit student data significantly better and change mastery-learning practice decisions for many students¶

Claim · All claims
Evidence · 1 study · associational r2 · q2
Currency · newest study 2012 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2015: tool and model accuracy changes quickly

Subclaims¶

q2 i? Fitting Knowledge Tracing parameters per student gave significantly better fit than a generic model (p < 0.0001) and would substantially alter practice-opportunity decisions for over 20% of students. → Kenneth R. Koedinger 2012

Evidence¶

Kenneth R. Koedinger 2012¶

Kenneth R. Koedinger, Emma Brunskill, Ryan S.J.d. Baker, Elizabeth A. McLaughlin, and John Stamper. (2012). New Potentials for Data-Driven Intelligent Tutoring System Development and Optimization. http://learnlab.org/datashop

q2 · i? · associational · r2

Retrospective policy evaluation on ASSISTments data from over 200 students compared generic versus individually fit KT parameters. The article reports "over 20% of students would be expected to be given at least twice as many expected practice opportunities" under the generic model, and a likelihood ratio test of p< 0.0001 favoring individual fit.

"over 20% of students would be expected to be given at least twice as many expected practice opportunities if the generic model was used compared to if their individual parameters were used, and 17% of students would be at <= 60% probability of mastery (compared to the threshold of 95%) when the generic model would declare the students as having mastered the material."

Discussion¶

Related Claims¶

• The exponential functional form of the BKT Markov chain calls into question the popular practice of fitting that model form to student data — related
• The HMM form of BKT, solved analytically, is a three-parameter exponential in opportunity number — related
• The identifiability problem of the BKT HMM arises because combinations of P(G) and P(L0) with the same product A give identical functional forms — related
• Standard knowledge tracing models in use do not allow individual learning rates or individualized initial knowledge — related