Variation across schools in principal-assigned professional practice ratings suggests principals' standards may not be fully consistent across schools¶

Claim · All claims
Evidence · 1 study · associational r? · q2
Currency · newest study 2014 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2017: tool and model accuracy changes quickly

Subclaims¶

q2 i? Between-school variation in professional practice ratings assigned by principals suggests principals' standards might not be fully consistent across schools. → Duncan Chaplin 2014

Evidence¶

Duncan Chaplin 2014¶

Duncan Chaplin, Brian Gill, Allison Thompkins, Hannah Miller. (2014). Professional Practice, Student Surveys, and Value-Added: Multiple Measures of Teacher Effectiveness in the Pittsburgh Public Schools. Regional Educational Laboratory Mid-Atlantic, Institute of Education Sciences. https://ies.ed.gov/ncee/edlabs

q2 · i? · associational · r?

Analysis of the principal-assigned professional practice rating found "Variation across schools" on that rating, which the authors read as a sign that principals' standards might not be fully consistent across schools. No variance statistics are printed in the available text.

"Variation across schools on the professional practice rating (which is assigned to teachers by the school principal) suggests that principals’ standards might not be fully consistent across schools"

Discussion¶

Related Claims¶

• The study used two sets of analyses—rating stability and the stable component's link to future contributions—to simulate each measure's predictive accuracy — related
• Professional practice ratings, student surveys, and value-added measures each have the potential to differentiate teacher performance in Pittsburgh Public Schools — related
• Component measure ratings had low to moderate positive correlations with each other, suggesting the components measure distinct dimensions of principal performance — related
• Fairness of overall observation scores may not be substantially compromised by principals using different sets of rubric components — related
• Most evaluators rated consistently, but about 3 percent were muted and 5 percent erratic, and about 10 percent of a sampled subset were extremely severe or lenient — related
• Principals in the pilot did not typically use all 22 rubric components when observing teachers — related
• Differences between principal and observer ratings at the high end of the scale were explained largely by teachers' previous evaluation ratings — related
• Most variation in teacher evaluation scores is within schools rather than between schools — related