Most school leaders received scores in the top two performance categories on FFL-measured practices¶

Claim · All claims
Evidence · 2 studies (3 entries) · 2 associational r2 · q2
Currency · newest study 2016 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2021: baselines and conditions change

Subclaims¶

q2 i? Score variation in the FFL was limited, with most school leaders earning scores in the top two performance categories in each measured leadership practice. → McCullough 2016

q2 i? On each measured leadership practice, most school leaders earned scores in the top two of the four possible performance levels. → Bing-ru Teh 2014

Evidence¶

McCullough 2016¶

McCullough, M., Lipscomb, S., Chiang, H., Gill, B., & Cheban, I. (2016). Measuring School Leaders' Effectiveness: Final Report from a Multiyear Pilot of Pennsylvania's Framework for Leadership. REL Mid-Atlantic, IES. https://ies.ed.gov/ncee/edlabs

q2 · i? · associational · r2

Pilot-year rating data from Pennsylvania's FFL show limited score variation: "Most school leaders received scores in the top two performance categories in each practice measured by the Framework for Leadership."

"Most school leaders received scores in the top two performance categories in each practice measured by the Framework for Leadership."

McCullough 2016 (2)¶

McCullough, M., Lipscomb, S., Chiang, H., Gill, B., & Cheban, I. (2016). Measuring School Leaders' Effectiveness: Final Report from a Multiyear Pilot of Pennsylvania's Framework for Leadership. REL Mid-Atlantic, IES. https://ies.ed.gov/ncee/edlabs

q2 · i? · associational · r2

The report's summary states that "in both pilot years, variation in scores was limited, with most school leaders scoring in the upper third of the rating scale."

"However, in both pilot years, variation in scores was limited, with most school leaders scoring in the upper third of the rating scale."

Bing-ru Teh 2014¶

Bing-ru Teh, Hanley Chiang, Stephen Lipscomb, Brian Gill. (2014). Measuring School Leaders' Effectiveness: An Interim Report from a Multiyear Pilot of Pennsylvania's Framework for Leadership. REL Mid-Atlantic, Institute of Education Sciences. https://www.mathematica.org/publications/measuring-school-leaders-effectiveness-an-interim-report-from-a-multiyear-pilot-of-pennsylvanias

q2 · i? · associational · r2

Score-variation analysis of the 2012/13 pilot, characterizing the percentages of school leaders earning scores in different portions of the rating scale. The finding of clustering in the top two levels suggests limited differentiation among leaders.

"On each measured leadership practice most school leaders earned scores in the top two of four possible performance levels."

Discussion¶

Related Claims¶

• The FFL shows good internal consistency across its full scale and its four categories of leadership practices — related
• A multiyear pilot study analyzed the score variation, internal consistency, score stability, and concurrent validity of Pennsylvania's Framework for Leadership — related
• School leaders who scored higher in one category of FFL leadership practices tended to score higher in the other categories — related
• School leaders who scored higher in one FFL category tended to score higher in the other categories — related
• Rating distributions and FFT-VAM relationships were broadly similar across the two pilot phases, though top-two-category ratings decreased somewhat in Pittsburgh — related
• The report concludes more validity evidence is needed before using FFL scores to identify effective and ineffective school leaders — related
• Performance tasks did little to differentiate teachers, with most receiving high scores and over a third receiving default scores due to missing data — related
• FFT-based teacher ratings in Pennsylvania's pilot were overwhelmingly concentrated in the top two performance categories, with less than 0.1 percent rated failing — related
• Principal practice ratings varied across the score range in New Jersey's pilot year, but most principals received ratings of effective or highly effective — related