A lottery-based randomized controlled trial of KIPP middle schools found statistically significant positive effects on state math and reading tests¶

Claim · All claims
Evidence · 1 study · causal r3 · q3
Currency · newest study 2018 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? In the third follow-up year after random assignment, KIPP lottery winners outperformed lottery losers on state standardized mathematics (+7 improvement index, 455 students) and reading (+6, 458 students) tests. → Knowledge Is Power Program (KIPP) 2018

Evidence¶

Knowledge Is Power Program (KIPP) 2018¶

Knowledge Is Power Program (KIPP). (2018). What Works Clearinghouse Intervention Report: Charter Schools. https://ies.ed.gov/ncee/wwc/InterventionReport/688

q3 · i? · causal · r3

Tuttle et al. (2015) randomized controlled trial: 891 lottery applicants (459 offered admission, 432 not offered) across 16 oversubscribed KIPP middle schools in 12 states and DC. Table A1 reports mathematics +7 (455 students) and English language arts +6 (458 students), both statistically significant. Meets WWC standards without reservations. No standardized effect size is printed, so impact is null.

"The authors reported, and the WWC confirmed, a positive and statistically significant difference between KIPP students and the comparison students for the middle school lottery sample measured in the third follow-up year after KIPP entry."

Discussion¶

Related Claims¶

• KIPP shows no discernible effects on high school student progression (graduation within 4 years) — related
• KIPP charter schools have positive effects on English language arts achievement for middle and high school students — a broader claim this one bears on
• KIPP charter schools have positive effects on mathematics achievement for middle and high school students — a broader claim this one bears on
• A randomized controlled trial found TFA secondary math teachers outperformed non-TFA comparison teachers in mathematics achievement — related
• In the vast majority of 22 KIPP middle schools, impacts on students' state assessment scores in math and reading are positive, statistically significant, and educationally substantial — related
• The Kauffman School's achievement impacts are higher than the average impacts found in studies of other charter schools — related
• KIPP elementary schools have positive, statistically significant, and educationally meaningful impacts on three of four measures of reading and mathematics skills — related
• For students continuing from KIPP middle schools to KIPP high schools, achievement impacts are not statistically significant — related
• Average KIPP middle school impacts were positive and statistically significant throughout the 10-year study period, though higher in earlier years than recent years — related
• KIPP middle schools have positive, statistically significant, and educationally meaningful impacts on student achievement in math, reading, science, and social studies — a broader claim this one bears on
• KIPP charter middle schools produce positive, statistically significant test-score effects in reading, math, science, and social studies — a broader claim this one bears on
• A lottery-based randomized experimental analysis confirms KIPP middle schools' positive impacts — possibly the same claim (merge candidate)
• KIPP outcomes were measured with state tests, a nationally normed assessment including higher-order thinking measures, and student- and parent-reported behaviors — related
• The Fisher Fellowship selection instrument is reliable, though three of 34 items had relatively low levels of one type of reliability — related
• Impacts of charter middle schools on achievement varied considerably across schools and students — related
• An earlier report of the national KIPP evaluation described strong positive achievement impacts in math and reading for 22 KIPP middle schools — related
• Prior research on KIPP suggested positive achievement impacts but was limited by small samples or methodological weaknesses — a broader claim this one bears on
• On average, lottery-based charter middle schools are neither more nor less successful than traditional public schools in improving student achievement — reports the opposite
• Students in single-discipline courses begin the year with higher achievement, roughly three or four points across subjects — related
• KIPP middle school offers did not produce higher five-year four-year graduation or on-track rates than no offer — related
• The KIPP lottery-based randomized controlled trial yielded treatment and control groups similar on observable and unobservable characteristics — related
• A lottery-based randomized controlled trial at Lied replicated the positive, statistically significant impact on 6th graders' mathematics achievement found by the QED — related
• Successor Prep school outcomes appeared similar to other KIPP schools, though substantial differences cannot be ruled out — related
• Oversubscription is prevalent among charter middle schools, enabling lottery-based evaluation — related
• In the earlier National Evaluation of Charter Middle Schools, admission to about 30 charter middle schools produced no significant average effects on achievement, though some schools improved achievement — related
• Two student-level randomized studies in California, Maine, and Minnesota high schools found significant positive impacts on NWEA MAP reading (average effect size +0.09) — related