Pair implementation description with quasi-experimental impact estimation in program evaluation¶

Strategy · All strategies
Evidence · 1 claim (1 mixed) · 1 study (1 causal), q2 · 0 of 1 report an effect size · 1 claim rests on one study

Description¶

The study combines two evaluation activities: it "describes program implementation across sites" and "employs a quasi-experimental design through propensity score matching methods" for impact estimation on key student outcomes. This pairing lets evaluators document how a multi-site program operates while estimating its effects on downstream outcomes such as college graduation and professional school matriculation.

Design Implications¶

Context¶

Requirements¶

• Implementation data across all program sites plus comparison data suitable for propensity score matching

Constraints¶

• Quasi-experimental rather than randomized design; outcomes are downstream (graduation, matriculation)

Target Learners¶

• Undergraduate program participants from disadvantaged backgrounds

Target Learning Goals¶

• College completion and entry to medical or dental school

Claims¶

• A matched-comparison analysis found no effect of the TenMarks summer program on NWEA MAP math scores [~W]

Related Strategies¶

Examples¶

-

Key Sources¶

• Clemencia Cosentino, Cecilia Speroni, Margaret Sullivan, Raúl Torres. (2015). Impact Evaluation of the RWJF Summer Medical and Dental Education Program (SMDEP). Washington, DC: Mathematica Policy Research. https://www.mathematica.org/publications/impact-evaluation-of-the-rwjf-summer-medical-and-dental-education-program-smdep