Principals were far more likely than trained external observers to rate the same instruction Distinguished rather than Proficient¶

Claim · All claims
Evidence · 1 study (2 entries) · associational r2 · q2
Currency · newest study 2011 · checked against its sources 2026-10-20 (automated, 2 of 2 entries) · not reviewed by a person · overdue for a recheck since 2014: tool and model accuracy changes quickly

Subclaims¶

q2 i? Across all components, principals were significantly more likely than observers to call Proficient-level practice Distinguished. → Sartain 2011 q2 i? 52 percent of simultaneously observed teachers received at least one Distinguished rating from principals versus 24 percent from observers. → Sartain 2011 (2)

Evidence¶

Sartain 2011¶

Sartain, L., Stoelinga, S. R., & Brown, E. R., with Luppescu, S., Kapadia Matsko, K., Miller, F. K., Durwood, C. E., Jiang, J. Y., & Glazer, D. (2011). Rethinking Teacher Evaluation in Chicago: Lessons Learned from Classroom Observations, Principal-Teacher Conferences, and District Implementation. Consortium on Chicago School Research at the University of Chicago. https://consortium.uchicago.edu/publications/rethinking-teacher-evaluation-chicago-lessons-learned-classroom-observations-principal

q2 · i? · associational · r2

Statistical comparison of 501 matched principal-observer ratings of the same lesson found principals "significantly more likely to call practice Distinguished when observers rated the same instruction as Proficient." Figure 3 shows an odds ratio of 6.18, significant at the .01 level; the article prints no standardized effect size for this contrast.

"Across all components, principals were also significantly more likely to call practice Distinguished when observers rated the same instruction as Proficient"

Sartain 2011 (2)¶

Sartain, L., Stoelinga, S. R., & Brown, E. R., with Luppescu, S., Kapadia Matsko, K., Miller, F. K., Durwood, C. E., Jiang, J. Y., & Glazer, D. (2011). Rethinking Teacher Evaluation in Chicago: Lessons Learned from Classroom Observations, Principal-Teacher Conferences, and District Implementation. Consortium on Chicago School Research at the University of Chicago. https://consortium.uchicago.edu/publications/rethinking-teacher-evaluation-chicago-lessons-learned-classroom-observations-principal

q2 · i? · design · r2

Descriptive ratings data from the matched-observation reliability sample of 257 teachers: "52 percent received at least one Distinguished rating from principals, but only 24 percent received a Distinguished rating from observers." Overall, principals gave 17 percent Distinguished ratings versus 3 percent for observers.

"Of the 257 teachers observed simultaneously by the principal and observer, 52 percent received at least one Distinguished rating from principals, but only 24 percent received a Distinguished rating from observers."

Discussion¶

Related Claims¶

• Principals rated some instruction components inconsistently, rating communicating with students, using assessment in instruction, and organizing physical space lower than observers, and student engagement higher — reports the opposite
• Principal-teacher conferences were perceived as more reflective and objective than past evaluations, but many principals lacked the instructional coaching skills for deep discussions of practice — related
• Principals and trained external observers gave aggregate-consistent Danielson Framework ratings in matched observations — reports the opposite
• Principals inflated ratings at the high end of the scale, more often identifying instruction as distinguished than observers — possibly the same claim (merge candidate)
• Principals rated teaching practice reliably at the low and middle ends of the rating scale, and most principals' severity matched trained observers — reports the opposite
• Actual classroom observation practice often differs from best practice, with principal-teacher perception gaps on measures and observer qualifications — related
• Most evaluators rated consistently, but about 3 percent were muted and 5 percent erratic, and about 10 percent of a sampled subset were extremely severe or lenient — related
• Differences between principal and observer ratings at the high end of the scale were explained largely by teachers' previous evaluation ratings — related