Implementation fidelity of cognitive science principles mediates the CS intervention's effect on student achievement, with no significant direct effect remaining¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r2 · q3
Currency · newest study 2017 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? Teachers in the CS condition reported overall implementation 0.63 standard deviations higher than control teachers (p = .000). → Desimone and Hill 2017 q3 i? Each one standard deviation increase in implementation frequency corresponds to a 0.05 standard deviation increase in student achievement (p = .06). → Desimone and Hill 2017 q3 i? The intervention effect is fully mediated: there was no significant direct effect of the CS intervention after inclusion of implementation and content knowledge. → Desimone and Hill 2017 (2)

Evidence¶

Desimone and Hill 2017¶

Laura M. Desimone, Kirsten Lee Hill. (2017). Inside the Black Box: Examining Mediators and Moderators of a Middle School Science Intervention. Educational Evaluation and Policy Analysis. https://doi.org/10.3102/0162373717697842

q3 · i? · causal · r2

Structural equation model using data from 10,281 students and 124 teachers in a clustered RCT. The article reports teachers in the CS condition "report levels of overall implementation 0.63 standard deviations higher than teachers in the control condition" and a 0.05 SD achievement increase per SD of implementation (p = .06); no standardized effect size is printed for these paths.

"teachers in the CS condition report levels of overall implementation 0.63 standard deviations higher than teachers in the control condition, even after controlling for major and years of experience (p = .000)"

Desimone and Hill 2017 (2)¶

Laura M. Desimone, Kirsten Lee Hill. (2017). Inside the Black Box: Examining Mediators and Moderators of a Middle School Science Intervention. Educational Evaluation and Policy Analysis. https://doi.org/10.3102/0162373717697842

q3 · i? · causal · r2

SEM results section: the fitted model (CFI = 0.873, RMSEA = 0.019) showed the treatment had a direct but nonsignificant effect on achievement, and the article concludes the effect "is fully mediated by these paths."

"there was no significant direct effect of the CS intervention after inclusion of implementation and content knowledge in our model, suggesting that the effect is fully mediated by these paths"

Discussion¶

Related Claims¶

• Classroom prior achievement moderates the treatment-implementation relationship: implementation gaps between high- and low-achieving classrooms appear only in the CS condition — related
• Implementation fidelity was high and treatment classrooms differed from controls in text complexity, task openness, and instructional time — related
• CS-condition teachers had lower content knowledge than control teachers, yet content knowledge was positively related to student achievement — related
• Teacher experience and STEM major predict content knowledge but not implementation of the intervention — related