Matched comparison group design for charter school impact evaluation¶

Element · All elements
Evidence · 3 claims (3 for) · 3 studies (3 causal), q2–q3 · 0 of 3 report an effect size · 2 claims rest on one study

Description¶

The evaluation method used for this report: a quasi-experimental, matched comparison group design estimating the Kauffman School's impacts on student achievement, attendance, and suspensions. The report states, "This paper uses a quasi-experimental, matched comparison group design to estimate the impacts of the Kauffman School on student achievement, attendance, and suspensions."

Design Implications¶

Context¶

Requirements¶

• A matched comparison group of comparable students to estimate impacts absent random assignment

Constraints¶

• Quasi-experimental rather than randomized; impacts are estimated relative to matched comparison schools

Target Learners¶

• K-12 charter school students

Target Learning Goals¶

• Estimating school-level impacts on achievement growth, attendance, and suspensions

Claims¶

• The Kauffman School had substantial positive, statistically significant impacts on student achievement in mathematics, reading, and science during its first three years of operation [+W]
• The Kauffman School raised student attendance by approximately 1 percentage point relative to comparison students [+W]
• The Kauffman School suspends students at a substantially higher rate than other schools in Kansas City [+W]

Related Elements¶

Examples¶

• Use a matched comparison group design plus qualitative mechanism exploration to evaluate a new charter school's educational and behavioral impacts
• Evaluate charter school impacts across college enrollment, graduation, attendance, and suspensions over multiple years
• Pair implementation description with quasi-experimental impact estimation in program evaluation

Key Sources¶

• Matthew Johnson, Eric Lundquist, Alicia Demers, Cleo Jacobs Johnson, Claudia Gentile. (2016). Ewing Marion Kauffman School Evaluation Impact Report: Year 3. Mathematica Policy Research. https://www.mathematica.org/publications/ewing-marion-kauffman-school-evaluation-impact-report-year-3