Use a matched comparison group design with qualitative mechanism exploration to evaluate charter school impacts¶

Strategy · All strategies
Evidence · 1 claim (1 for) · 1 study (1 causal), q1 · 0 of 1 report an effect size · 1 claim rests on one study

Description¶

The paper's evaluation approach pairs a quantitative impact estimate with qualitative description of school operations. It "used a matched comparison group design to estimate the impacts of the Kauffman School on student achievement, attendance, and suspensions", and also "used surveys, interviews, focus groups, and classroom observations to describe the hallmarks and operations" and "explore possible mechanisms for its effects".

Design Implications¶

Context¶

Requirements¶

• A matched comparison group for the impact analysis, plus access to stakeholders and classrooms for surveys, interviews, focus groups, and observations
• A matched comparison group must be available for impact estimation
• Qualitative data collection with stakeholders to explore mechanisms

Constraints¶

• A matched comparison group design yields quasi-experimental rather than randomized estimates

Target Learners¶

• students from low-income neighborhoods in Kansas City, Missouri

Target Learning Goals¶

• estimating school-level impacts on achievement, attendance, and suspensions
• Academic achievement in mathematics, reading, and science; attendance; behavior (suspensions)

Claims¶

• Qualitative evidence indicates the Kauffman School's hallmarks were implemented faithfully and stakeholders believed its methods positively influenced students' attitudes and performance [+W]

Related Strategies¶

• Evaluate charter school impacts by matching students to demographically and academically similar comparison students

Examples¶

-

Key Sources¶

• Matthew Johnson, Cleo Jacobs Johnson, Scott Richman, Alicia Demers, Claudia Gentile, Eric Lindquist. (2016). The Educational and Behavioral Impacts of the Ewing Marion Kauffman Charter School. Mathematica Policy Research, Working Paper 43. https://www.mathematica.org/publications/the-educational-and-behavioral-impacts-of-the-ewing-marion-kauffman-charter-school
• Matthew Johnson, Cleo Jacobs Johnson, Scott B. Richman, Alicia Demers, Claudia Gentile, Eric Lundquist. (2017). The Educational and Behavioral Impacts of the Ewing Marion Kauffman Charter School. Journal of School Choice, vol. 11, issue 3. https://www.mathematica.org/publications/ja-the-educational-and-behavioral-impacts-of-the-ewing-marion-kauffman-charter-school