On-treatment fidelity analysis found low-fidelity schools improved outcomes relative to non-implementers, but the pattern was inconsistent across higher fidelity groups¶

Claim · All claims
Evidence · 1 study · causal r2 · q2
Currency · newest study 2015 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Schools with a low fidelity rating had improved outcomes relative to schools that had not implemented the programme, but this was not consistent across the Medium and High fidelity groups. → Quest: Evaluation report and Executive summary 2015

Evidence¶

Quest: Evaluation report and Executive summary 2015¶

Quest: Evaluation report and Executive summary. (2015). Education Endowment Foundation. https://eric.ed.gov/?id=ED581287

q2 · i? · causal · r2

On-treatment analysis of 931 observations using lesson-observation fidelity ratings (0–3 scale). Table 3.10 shows Low rating coef. 0.38*** (s.e. 0.06), Medium 0.17 (s.e. 0.07), High -0.05 (s.e. 0.07), against a not-implemented reference category.

"The results suggest that schools with a low fidelity rating had improved outcomes relative to those who had not implemented the programme successfully at all (Table 3.10). However, this finding was not consistent across the two higher fidelity groups (Medium and High)."

Discussion¶

Related Claims¶

• Personalized Learning Effects Vary With Fidelity — related
• Teachers implemented the PACT components with generally high fidelity, and PACT elements appeared only at low rates in comparison classrooms — related
• Few, if any, schools implemented Quest as designed, with low implementation possibly diluting any impact — related