Completing more skill levels is associated with higher Checkpoint Quiz post-test accuracy, with positive coefficients for Levels 1, 2, and 4¶

Claim · All claims
Evidence · 1 study · causal r2 · q2
Currency · newest study 2021 · not yet checked against its sources · not reviewed by a person
Evidence quality · No study behind this claim is a peer-reviewed article with a DOI: it rests on a source with no registered publication record (a web page, book or unpublished report). It is kept because 2 design pages cite it; treat it as a lead to check, not an established finding.

Subclaims¶

q2 i? Average post-test item accuracy increases linearly with every skill-level completed, and the logistic regression shows the probability of answering a post-test item correctly increases with every additional lesson in Levels 1, 2, and 4. → Lucy Portnoff 2021

Evidence¶

Lucy Portnoff 2021¶

Lucy Portnoff, Erin Gustafson, Klinton Bicknell and Joseph Rollinson. (2021). Methods for Language Learning Assessment at Scale: Duolingo Case Study. Proceedings of The 14th International Conference on Educational Data Mining (EDM21). https://educationaldatamining.org/edm2021/

q2 · i? · causal · r2

Logistic regression on four months of Checkpoint Quiz data predicted post-test accuracy on pre-test-incorrect items, controlling for item, user, course, other session types, prior proficiency, and subscriber status. The article reports "the probability of answering a post-test item correctly increases with every additional lesson in Levels 1, 2, and 4"; no effect sizes are printed.

"We observed that the probability of answering a post-test item correctly increases with every additional lesson in Levels 1, 2, and 4. Level 3 has a negative coefficient, but this is likely an artifact of variable suppression"

Discussion¶

Related Claims¶

• A regression discontinuity design on Review Exercise data supports a causal link between leveling up and higher assessment accuracy, at least for the first level-up — related
• Leveling up lessons preceding the source lesson improves Review Exercise accuracy, indicating transfer of learning benefits across lessons within a skill — a narrower finding that bears on this claim
• Repeated successful retrieval during learning predicted final recall in both experiments — related