Rating distributions and FFT-VAM relationships were broadly similar across the two pilot phases, though top-two-category ratings decreased somewhat in Pittsburgh¶

Claim · All claims
Evidence · 1 study (2 entries) · associational r2 · q2
Currency · newest study 2015 · checked against its sources 2026-10-20 (automated, 2 of 2 entries) · not reviewed by a person

Subclaims¶

q2 i? More than 90 percent of teachers were rated in the top two categories in both phases, but the fraction in the top two categories decreased somewhat in Pittsburgh, the district contributing the most teachers. → Stephen Lipscomb 2015 q2 i? Correlations between parts of the FFT and VAM scores were almost always positive but below 0.30 in both phases, with the lowest 2011-2012 correlations improving slightly in 2012-2013. → Stephen Lipscomb 2015 (2)

Evidence¶

Stephen Lipscomb 2015¶

Stephen Lipscomb, Jeffrey Terziev, Duncan Chaplin. (2015). Measuring Teachers' Effectiveness: A Report from Phase 3 of Pennsylvania's Pilot of the Framework for Teaching. Washington, DC: Mathematica Policy Research. https://www.mathematica.org/publications/measuring-teachers-effectiveness-a-report-from-phase-3-of-pennsylvanias-pilot-of-the-framework

q2 · i? · associational · r2

Comparison of 2012-2013 results with 2011-2012 data from the previous pilot phase. The report found more than 90 percent in the top two categories in both phases, with a somewhat smaller fraction in Pittsburgh.

"More than 90 percent of teachers were rated in the top two performance categories in both phases, although the fraction of ratings in the top two categories decreased somewhat in Pittsburgh (which contributed more teachers to the pilot than any other district)."

Stephen Lipscomb 2015 (2)¶

Stephen Lipscomb, Jeffrey Terziev, Duncan Chaplin. (2015). Measuring Teachers' Effectiveness: A Report from Phase 3 of Pennsylvania's Pilot of the Framework for Teaching. Washington, DC: Mathematica Policy Research. https://www.mathematica.org/publications/measuring-teachers-effectiveness-a-report-from-phase-3-of-pennsylvanias-pilot-of-the-framework

q2 · i? · associational · r2

Cross-phase comparison of FFT-VAM correlations. The report states correlations were "almost always positive but also below 0.30 in both phases", with the lowest 2011-2012 correlations improving slightly the next year.

"The correlations between parts of the FFT and VAM scores were almost always positive but also below 0.30 in both phases. The lowest correlations in 2011–2012 improved slightly in 2012–2013."

Discussion¶

Related Claims¶

• FFT-based teacher ratings in Pennsylvania's pilot were overwhelmingly concentrated in the top two performance categories, with less than 0.1 percent rated failing — related
• FFT scores correlated positively with value-added measures, with domain correlations ranging from 0.19 to 0.22 and generally statistically significant — related
• Most school leaders received scores in the top two performance categories on FFL-measured practices — related
• FFT scores showed acceptable-to-good internal consistency, with the overall PPR score more internally consistent than any single domain score — related
• Performance tasks did little to differentiate teachers, with most receiving high scores and over a third receiving default scores due to missing data — related
• Districts rate ISAT alignment to state learning standards highest among five rated areas, but fewer than half use the top two categories, and month administered and speed of reporting rate lowest — related