SMS nudges targeting socio-emotional skills increased student-level standardized test scores during remote learning, by 0.21 s.d. in Portuguese and 0.19 s.d. on the summary measure¶

Claim · All claims
Evidence · 1 study (3 entries) · causal r3 · q3 · i0–i1
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i1 Nudges increased Portuguese standardized test scores by 0.21 s.d. (90% CI: [0.06, 0.35]) relative to control. → Lichand 2024 q3 i0 On the pre-registered summary measure combining Portuguese and math, the effect was 0.19 s.d. (90% CI: [0.03, 0.35]). → Lichand 2024 (2) q3 i0 The math effect was smaller and only imprecisely estimated (0.10 s.d.; 90% CI: [-0.05, 0.25]). → Lichand 2024 (3)

Evidence¶

Lichand 2024¶

Lichand, G., Christen, J., & Van Egeraat, E. (2024). Neglecting students’ socio-emotional skills magnified learning losses during the pandemic. npj Science of Learning. https://doi.org/10.1038/s41539-024-00235-9

q3 · i1 · causal · r3

Cluster-randomized control trial in Goiás, Brazil (18,256 students, 87 schools), ITT OLS estimates on student-level test scores from an in-person Q2/2021 assessment. The printed effect size is 0.21 s.d. (90% CI: [0.06, 0.35]) on Portuguese test scores, a small standardized effect.

"Column 1 shows that nudges increased Portuguese standardized test scores by 0.21 s.d. (90% CI: [0.06, 0.35])."

Lichand 2024 (2)¶

Lichand, G., Christen, J., & Van Egeraat, E. (2024). Neglecting students’ socio-emotional skills magnified learning losses during the pandemic. npj Science of Learning. https://doi.org/10.1038/s41539-024-00235-9

q3 · i0 · causal · r3

Pre-registered specification on the summary measure of Portuguese and math standardized test scores in the same cluster-RCT; the printed effect size is "0.19 s.d. in column 3; 90% CI: [0.03, 0.35]", just below the small-effect threshold.

"the effect of nudges is still very large and signi ficantly estimated in the pre- registered speci fication (0.19 s.d. in column 3; 90% CI: [0.03, 0.35])"

Lichand 2024 (3)¶

Lichand, G., Christen, J., & Van Egeraat, E. (2024). Neglecting students’ socio-emotional skills magnified learning losses during the pandemic. npj Science of Learning. https://doi.org/10.1038/s41539-024-00235-9

q3 · i0 · causal · r3

Math outcome in the same student-level ITT analysis; the printed effect size is "0.10 s.d. in column 2; 90% CI: [ −0.05, 0.25]", below the small-effect threshold and not statistically significant.

"When it comes to math, nudges also increased test scores, but the effect size was smaller and only imprecisely estimated (0.10 s.d. in column 2; 90% CI: [ −0.05, 0.25])"

Discussion¶

Related Claims¶

• Text-message nudges achieving impacts of the same order as remote remedial instruction indicates motivation itself carries much of remote-learning effectiveness — related
• Nudges significantly decreased the share of schools with sub-proficient average test scores, from 17.1% to 8.4% — related
• Nudges prevented 24.2% of learning losses in Portuguese and 7.5% in math at school level, though imprecisely estimated — related
• Message content mattered: loss framing and avoiding social pressure produced significantly larger test-score effects, ruling out experimenter-demand explanations — related
• The intervention increased student engagement with remote activities and motivation to return to in-person classes — related
• Treatment effects were larger for previously higher-performing students and for schools with prior online activities, consistent with complementarity between curricular content and socio-emotional skills — related
• Targeting students directly (versus caregivers) produced significantly larger nudge effects on test scores (0.31 vs. 0.08 s.d.) — related
• Parents-only SMS/audio nudges produced statistically non-significant learning gains of d = 0.081 in rural Cote d'Ivoire — related