There is little prior evidence on the validity of principal evaluation measures, and this is the first study linking practice ratings to credible value-added measures¶

Claim · All claims
Evidence · 1 study · associational r? · q1
Currency · newest study 2016 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2019: tool and model accuracy changes quickly

Subclaims¶

q1 i? The authors assert that little evidence exists on the validity of principal evaluation measures and that their study is the first to find such a correlation with credible value-added measures. → Moira McCullough 2016

Evidence¶

Moira McCullough 2016¶

Moira McCullough, Stephen Lipscomb, Hanley Chiang, Brian Gill. (2016). Do Principals' Professional Practice Ratings Reflect Their Contributions to Student Achievement? Evidence from Pennsylvania's Framework for Leadership. Working Paper 46. Cambridge, MA: Mathematica Policy Research. https://www.mathematica.org/publications/do-principals-professional-practice-ratings-reflect-their-contributions-to-student-achievement

q1 · i? · associational · r?

Authors' framing statement about the research gap: although states and districts have begun evaluating principals, "there is little evidence on the validity of principal evaluation measures." This is the authors' interpretation, not a tested result.

"This is the first study to find evidence that ratings of principals' professional practice are correlated with credible measures of principals' contributions to student achievement."

Discussion¶

Related Claims¶

• FFL evaluation scores are significantly and positively correlated with principals' contributions to student achievement (value-added) — related
• Principal practice ratings varied across the score range in New Jersey's pilot year, but most principals received ratings of effective or highly effective — related
• The study used two sets of analyses—rating stability and the stable component's link to future contributions—to simulate each measure's predictive accuracy — related
• FFL scores correlate more strongly with contributions to math achievement than to achievement in other subjects — related
• School median student growth percentiles were stable year to year even when the school changed principals — related
• Mathematica developed and estimated VAMs for use in Phase 1 of the Pennsylvania evaluation pilot — related
• School value added can reflect school-specific influences on student achievement growth that are outside principals' control — related
• Federal policy requirements drove growth modeling and teacher value-added evaluation under NCLB and RTTP — related
• Value-added models were used to estimate teacher and principal effectiveness measures in Charleston County School District in 2013-2014 — related
• Nearly all teacher survey domains and scales correlate small-to-moderately (0.26 to 0.46) with supervisors' ratings of principals' leadership — related
• The study assessed rating stability for 2,424 principals and predictive validity for 123 principals using Pennsylvania statewide data — related
• Higher FFL scores associate with larger value-added for middle school principals but not for elementary or high school principals — related