The Detroit Promise Path effect of 1.8 credits earned after two semesters sits at the 83rd percentile of the distribution of community college intervention effects, indicating a quite large effect¶

Claim · All claims
Evidence · 1 study · quant-synthesis r2 · q2
Currency · newest study 2023 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Benchmarked against THE-RCT, DPP's estimated 1.8-credit effect yields an adjusted empirical Bayes estimate of 1.6 credits, about 0.70 standard deviations above the mean and at the 83rd percentile after two semesters. → Michael J. Weiss 2023

Evidence¶

Michael J. Weiss 2023¶

Michael J. Weiss, Marie-Andrée Somers, & Colin Hill. (2023). Empirical Benchmarks for Planning and Interpreting Causal Effects of Community College Interventions. Journal of Postsecondary Student Success 3 (1). https://doi.org/10.33009/fsop_jpss132759

q2 · i? · quant-synthesis · r2

Illustrative interpretation in the Discussion applying the paper's benchmarks to the DPP RCT (Ratledge et al., 2021), whose estimated effect was 1.8 credits after two semesters. The article concludes the DPP effect "is quite large compared to effects of other evaluated interventions."

"This results in an adjusted empirical Bayes impact estimate of 1.6 credits earned, which is about 0.70 standard deviations above the mean effect, and at the 83rd percentile in the distribution of effects after two semesters."

Discussion¶

Related Claims¶

• Mean effects of community college interventions on cumulative credits earned increase over the first six semesters after random assignment, reaching 1.14 credits by semester 3 — related
• Mean effects of community college interventions on marginal (per-semester) enrollment and credits earned decrease over time, with decreasing variability — related
• Average effects of community college interventions on degree completion are small (0.9–1.7 percentage points by semesters 4–6) and benchmarking is limited because two outlier interventions drive the distribution — related
• For credits earned in semester 1, the estimated distribution of true intervention effects has a mean of 0.48 credits, a cross-intervention standard deviation of 0.55 credits, and a 75th percentile of 0.60 credits — related
• Cross-intervention variation in community college intervention effects is substantial: for first-semester credits earned, the true-effect standard deviation (0.55 credits) exceeds the mean effect (0.48 credits) — related
• Distributions of estimated intervention effects overstate the spread of true effects; adjusted empirical Bayes estimates reduce the spread by about 16% for cumulative semesters enrolled — related