The selection algorithm affects engagement: select and difficulty groups were more likely to return within 4–7 days than the random group, but select-group learners were more likely to stop using the app in the initial 2 days¶
Claim · All claims
Evidence · 1 study · causalr2·q2
Currency · newest study 2021 · not yet checked against its sources · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? Learners in the 'select' ('difficulty') group were 50.6% (47.6%) more likely, in median, to return to the app within 4–7 days than learners of the 'random' group, but select-group learners were also more likely to stop using the app in the initial 2 days. → Utkarsh Upadhyay 2021
Evidence¶
Utkarsh Upadhyay 2021¶
Utkarsh Upadhyay, Graham Lancashire, Christoph Moser and Manuel Gomez-Rodriguez. (2021). Large-scale randomized experiments reveals that machine learning-based instruction helps people memorize more effectively. npj Science of Learning 6:26. https://doi.org/10.1038/s41539-021-00105-8
q2 · i? · causal · r2
Engagement analysis from the randomized controlled trial, reported in the main text with details deferred to Supplementary Information. The article reports median return-likelihood percentages and notes the early-quit counter-effect; no standardized effect size is printed.
"learners of the ‘select’ (‘difficulty’) group were 50.6% (47.6%) more likely, in median, to return to the app within 4 –7 days than learners of the ‘random’ group. However, learners of the ‘select’ group were also more likely to stop using the app in the initial 2 days than those of the other groups."
Discussion¶
Related Claims¶
- Algorithm-selected learners remember content over ~69% longer, with median half-life increases of ~92% vs random and ~40% vs difficulty groups — related
- ML-based question selection (SELECT) lowers learners' normalized empirical forgetting rate relative to difficulty-ordered and random selection after controlling for review time and number of reviews — related
- Regression analysis shows the spacing effect holds in aggregate and the select group's advantage over other groups increases with number of reviews — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
select-algorithm-engagement-effects |
title |
The selection algorithm affects engagement: select and difficulty groups were more likely to return within 4–7 days than the random group, but select-group learners were more likely to stop using the app in the initial 2 days |
description |
The selection algorithm affects engagement: select and difficulty groups were more likely to return within 4–7 days than the random group, but select-group learners were more likely to stop using the app in the initia... |
status |
review |
evidence_strength |
weak |
newest_evidence |
2021 |
generated |
by process:wiki-ingest on 2026-10-26 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — utkarsh-upadhyay-2021 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.