Differences between principal and observer ratings at the high end of the scale were explained largely by teachers' previous evaluation ratings¶

Claim · All claims
Evidence · 1 study · associational r3 · q2
Currency · newest study 2011 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2014: tool and model accuracy changes quickly

Subclaims¶

q2 i? When teachers' previous evaluation ratings were added to the statistical models, much of the principal-observer rating variation disappeared and the high-end difference was no longer statistically significant. → Sartain 2011

Evidence¶

Sartain 2011¶

Sartain, L., Stoelinga, S. R., & Brown, E. R., with Luppescu, S., Kapadia Matsko, K., Miller, F. K., Durwood, C. E., Jiang, J. Y., & Glazer, D. (2011). Rethinking Teacher Evaluation in Chicago: Lessons Learned from Classroom Observations, Principal-Teacher Conferences, and District Implementation. Consortium on Chicago School Research at the University of Chicago. https://consortium.uchicago.edu/publications/rethinking-teacher-evaluation-chicago-lessons-learned-classroom-observations-principal

q2 · i? · associational · r3

Statistical re-analysis of the matched ratings showed that after controlling for prior checklist evaluation ratings, "much of the variation between principal and observer ratings disappeared," and the high-end difference "is no longer statistically significant." The authors offer two interpretations: principals may use historical knowledge or may rate more accurately.

"When we took into account teachers' previous evaluation ratings in the statistical models, much of the variation between principal and observer ratings disappeared."

Discussion¶

Related Claims¶

• Principals and trained external observers gave aggregate-consistent Danielson Framework ratings in matched observations — related
• Principals inflated ratings at the high end of the scale, more often identifying instruction as distinguished than observers — related
• Variation across schools in principal-assigned professional practice ratings suggests principals' standards may not be fully consistent across schools — related
• Principals rated some instruction components inconsistently, rating communicating with students, using assessment in instruction, and organizing physical space lower than observers, and student engagement higher — related
• Principals rated teaching practice reliably at the low and middle ends of the rating scale, and most principals' severity matched trained observers — related
• Principals were far more likely than trained external observers to rate the same instruction Distinguished rather than Proficient — related