Three of the five reviewed RCTs found no statistically significant effect of summer counseling on fall college enrollment (indeterminate effects)¶

Claim · All claims
Evidence · 1 study (3 entries) · causal r2–r3 · q3 · i0 negligible
Currency · newest study 2018 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i0 In New Mexico (Castleman et al., 2015a; 1,602 students), the enrollment effect was -0.01 (improvement index 0), not statistically significant. → Summer Counseling 2018 q3 i0 In the Bridgit cluster RCT (Castleman et al., 2015b; 3,281 students), enrollment effect size was 0.09 (+3), not statistically significant. → Summer Counseling 2018 (2) q3 i? In the texting and peer-mentoring RCT (Castleman & Page, 2015), both intervention types showed no statistically significant difference on fall enrollment. → Summer Counseling 2018 (3)

Evidence¶

Summer Counseling 2018¶

Summer Counseling. (2018). What Works Clearinghouse Intervention Report, Transition to College. U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. https://ies.ed.gov/ncee/wwc/InterventionReport/693

q3 · i0 · causal · r3

This New Mexico RCT assigned 1,602 college-admitted graduates to counselor outreach (based at the University of New Mexico or Albuquerque Public Schools) or comparison; the WWC table shows an enrollment effect size of -0.01 and p > .10.

"Castleman et al. (2015a) reported, and the WWC confirmed, no statistically significant difference between the intervention group and the comparison group on fall college enrollment. The WWC characterizes this study finding as an indeterminate effect."

Summer Counseling 2018 (2)¶

Summer Counseling. (2018). What Works Clearinghouse Intervention Report, Transition to College. U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. https://ies.ed.gov/ncee/wwc/InterventionReport/693

q3 · i0 · causal · r2

This cluster randomized trial in Tennessee and Missouri paired schools by prior-year college enrollment rate; six intervention and six comparison schools participated. The WWC table shows an effect size of 0.09 and p < .10, not statistically significant.

"Castleman et al. (2015b) reported, and the WWC confirmed, no statistically significant difference between the summer counseling group and the comparison group on fall college enrollment. The WWC characterizes this study finding as an indeterminate effect."

Summer Counseling 2018 (3)¶

Summer Counseling. (2018). What Works Clearinghouse Intervention Report, Transition to College. U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. https://ies.ed.gov/ncee/wwc/InterventionReport/693

q3 · i? · review · r3

This RCT across five sites compared a text-messaging campaign (2,524 intervention, 2,535 comparison) and peer mentoring (934 intervention, 1,272 comparison) against typical school supports; the appendix tables show effect sizes of 0.06 for both arms, p > .10, and improvement indices of +2 and +3.

"Castleman and Page (2015) reported, and the WWC confirmed, no statistically significant difference between the intervention groups and the comparison groups on fall college enrollment."

Discussion¶

Related Claims¶

• Summer counseling has mixed effects on college access and enrollment for recent high school graduates, with a domain-average improvement index of +5 percentile points — a broader claim this one bears on
• Summer counseling produces a small but statistically significant positive effect on credit accumulation and persistence (WWC rating: potentially positive effects) — related
• In the Castleman et al. (2012) RCT, summer counseling showed a substantively important positive effect on fall college enrollment despite non-significant test statistics — related