Randomized A/B tests of interventions encouraging higher-level study increased lessons completed per level by more than 10% and increased overall studying activity¶
Claim · All claims
Evidence · 1 study · causalr?·q3
Currency · newest study 2021 · not yet checked against its sources · not reviewed by a person
Evidence quality · No study behind this claim is a peer-reviewed article with a DOI: it rests on a source with no registered publication record (a web page, book or unpublished report). It is kept because 2 design pages cite it; treat it as a lead to check, not an established finding.
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q3 i? Design elements giving a visual stand-in for the levels system and a pop-up redirect prompting the next level produced >10% increases in lessons completed at each level beyond the foundational level. → Lucy Portnoff 2021
Evidence¶
Lucy Portnoff 2021¶
Lucy Portnoff, Erin Gustafson, Klinton Bicknell and Joseph Rollinson. (2021). Methods for Language Learning Assessment at Scale: Duolingo Case Study. Proceedings of The 14th International Conference on Educational Data Mining (EDM21). https://educationaldatamining.org/edm2021/
q3 · i? · causal · r?
Randomized controlled experiments (A/B tests) on the Duolingo platform tested visual stand-in design elements and a pop-up redirect button prompting learners to start the next level. The article reports ">10% increases in the number of lessons completed" per level and significantly more overall studying activity; no standardized effect size is printed.
"Randomized controlled experiments (i.e., A/B tests) introducing these changes showed >10% increases in the number of lessons completed in each level beyond the required foundational level and significantly more studying activity on the app overall."
Discussion¶
Related Claims¶
- In a randomized trial of SRL interventions in an online mathematics preparation course, group and time interacted significantly on SRL knowledge, self-efficacy and the SRL overall score, with Group TDP (training + diary + peer feedback) showing the most prominent gains — related
- SRL interventions in a four-week online mathematics preparation course did not significantly change the mathematics overall score (interaction marginal), while the mathematics focus score on self-chosen chapters showed a statistically significant group by time interaction — related
- Foundations can serve more people by identifying and supporting effective interventions that are ready to be scaled — related
- Students who use the ePSRL Management System meet the course completion deadline at a significantly higher rate than non-users — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
ab-tests-level-interventions-increase-lessons |
title |
Randomized A/B tests of interventions encouraging higher-level study increased lessons completed per level by more than 10% and increased overall studying activity |
description |
Randomized A/B tests of interventions encouraging higher-level study increased lessons completed per level by more than 10% and increased overall studying activity |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2021 |
generated |
by process:wiki-ingest on 2026-09-27 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — lucy-portnoff-2021 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.