Treatment-effect heterogeneity by engagement: first-day unsubscribers reenrolled 26 percentage points below control, while full-treatment students benefited whether or not they responded¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r2 · q3
Currency · newest study 2019 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? Students who unsubscribed on the first day reenrolled at a 26-percentage-point lower rate than the control group rate of 62% (p < .001). → Ross E. O'Hara 2019 q3 i? Students who remained subscribed benefited even without responding: 6 percentage points for never-responders (p = .004) and 24 percentage points for students who responded at least once (p < .001). → Ross E. O'Hara 2019 (2)

Evidence¶

Ross E. O'Hara 2019¶

Ross E. O'Hara, Betsy Sparrow. (2019). A Summer Nudge Campaign to Motivate Community College STEM Students to Reenroll. AERA Open. https://doi.org/10.1177/2332858419875715

q3 · i? · causal · r2

Exploratory heterogeneity analysis dividing the treatment group by treatment received: first-day unsubscribers (n = 74), partial treatment (n = 132), full treatment (n = 1,161). The authors interpret unsubscribing as flagging "students already on the cusp of withdrawal" and caution the result since only 5% unsubscribed immediately.

"For students who unsubscribed on the first day, we observed a 26-percentage-point lower rate of reenrollment compared with the control group rate of 62%, p < .001."

Ross E. O'Hara 2019 (2)¶

Ross E. O'Hara, Betsy Sparrow. (2019). A Summer Nudge Campaign to Motivate Community College STEM Students to Reenroll. AERA Open. https://doi.org/10.1177/2332858419875715

q3 · i? · causal · r2

Same heterogeneity analysis subdividing full-treatment students into never-responders (n = 898) and responders (n = 263). The authors note the responder advantage is likely largely attributable to unmeasured third variables driving both engagement and reenrollment.

"students who remained subscribed for the duration of the intervention were more likely to reenroll than the control group: A difference of 6 percentage points for students who never responded (p = .004) and a difference of 24 percentage points for students who responded at least once (p < .001)."

Discussion¶

Related Claims¶

• Summer-enrolled treatment students engaged more with the nudges than non-summer-enrolled students, while STEM and non-STEM students engaged similarly — related
• A 7-week summer text-message nudge campaign increased fall reenrollment among first-year community college students by 7 percentage points relative to control — a broader claim this one bears on
• The nudge intervention raised reenrollment by 10 percentage points among STEM-pathway students but had no significant effect on non-STEM students — related
• The nudge intervention raised reenrollment by 10 percentage points among summer-enrolled students but not significantly among students not enrolled in summer courses — related