Continue examining the FFL's statistical properties, especially score variation, during statewide implementation¶

Strategy · All strategies
Evidence · 3 claims (2 for, 1 mixed) · 3 studies (3 associational), q2–q3 · 0 of 3 report an effect size · 3 claims rest on one study

Description¶

The report recommends that as Pennsylvania's Framework for Leadership moves from pilot to statewide use, evidence on its statistical properties should keep being examined. It states: "As the FFL is implemented statewide, continued examination of evidence on its statistical properties, especially the variation in scores, is important." This follows pilot findings of limited score variation.

Design Implications¶

Context¶

Requirements¶

• Ongoing collection and analysis of FFL rating data during statewide implementation

Constraints¶

• Pilot-year score variation was limited, with most leaders in the upper third of the rating scale

Target Learners¶

• school principals and assistant principals

Target Learning Goals¶

• accurate evaluation of school leadership practices

Claims¶

• Higher FFL scores associate with larger value-added for middle school principals but not for elementary or high school principals [~W]
• Most variation in teacher evaluation scores is within schools rather than between schools [+W]
• School leaders' FFL scores are moderately stable year to year (correlation of .54) [+W]

Related Strategies¶

• Pilot statewide roster verification systems before full implementation

Examples¶

-

Key Sources¶

• McCullough, M., Lipscomb, S., Chiang, H., Gill, B., & Cheban, I. (2016). Measuring School Leaders' Effectiveness: Final Report from a Multiyear Pilot of Pennsylvania's Framework for Leadership. REL Mid-Atlantic, IES. https://ies.ed.gov/ncee/edlabs