Measurement error in the mediator reduces statistical power for mediator-achievement association estimates¶

Claim · All claims
Evidence · 1 study · theoretical r3 · q2
Currency · newest study 2009 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Measurement error reduces the non-centrality parameter, and hence statistical power, by lowering the model R2 values. → Schochet 2009

Evidence¶

Schochet 2009¶

Schochet, Peter Z. (2009). Do Typical RCTs of Education Interventions Have Sufficient Statistical Power for Linking Impacts on Teacher Practice and Student Achievement Outcomes? (NCEE 2009-4065). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. https://www.mathematica.org/publications/do-typical-rcts-of-education-interventions-have-sufficient-statistical-power-for-linking-impacts-on-teacher-practice-and-student-achievement-outcomes

q2 · i? · theoretical · r3

Analytical derivation using a standard measurement error model (Fuller 1987) for the OLS estimator. The report notes mediator measurement error "can be large, especially for classroom observation measures" collected by few raters over short intervals.

"Thus, measurement error reduces the non-yM M centrality parameter (and hence, statistical power) by lowering the model R2 values."

Discussion¶

Related Claims¶

• Mediator-outcome association analyses can identify teacher practice mediators most associated with student learning — related
• In clustered school-based RCTs, OLS mediator analyses yield precise teacher practice-achievement estimates only with about 150 to 200 study schools — related
• The IV approach to mediator analysis in clustered education RCTs has very little statistical power — related
• Estimating associations between student and teacher practice outcomes can examine the extent to which RCT data support a study's conceptual model — a broader claim this one bears on
• Multidimensional IRT scores that account for latent variable covariances over time produce better recovery of growth parameters in longitudinal growth models than sum scores and other IRT approaches — related
• The working paper examines the sensitivity and precision of teacher value-added estimates under specifications differing in student-level, peer-level, and double-lagged achievement controls — related