Principals and trained external observers gave aggregate-consistent Danielson Framework ratings in matched observations¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2010 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2013: tool and model accuracy changes quickly

Subclaims¶

q2 i? In aggregate, there was no significant difference between ratings given by principals and those given by trained external observers observing the same lessons. → Lauren Sartain 2010

Evidence¶

Lauren Sartain 2010¶

Lauren Sartain, Sara Ray Stoelinga, and Emily Krone. (2010). Findings From the First Year of the Excellence in Teaching Project in Chicago Public Schools: Rethinking Teacher Evaluation. Consortium on Chicago School Research, University of Chicago. http://ccsr.uchicago.edu/publications/Joyce_TE_yr1_finaldoc.pdf

q2 · i? · design · r2

Year-one reliability study using matched observations: external observers and school administrators observed the same classrooms simultaneously but rated independently; quantitative data included 277 matched observations across 44 pilot elementary schools. The study found "no significant difference between the ratings given by principals and those given by external observers" in aggregate.

"Considered in aggregate, there is no significant difference between the ratings given by principals and those given by external observers."

Discussion¶

Related Claims¶

• Principals rated some instruction components inconsistently, rating communicating with students, using assessment in instruction, and organizing physical space lower than observers, and student engagement higher — related
• Principals were far more likely than trained external observers to rate the same instruction Distinguished rather than Proficient — reports the opposite
• Principals rated teaching practice reliably at the low and middle ends of the rating scale, and most principals' severity matched trained observers — possibly the same claim (merge candidate)
• Differences between principal and observer ratings at the high end of the scale were explained largely by teachers' previous evaluation ratings — related
• Principals inflated ratings at the high end of the scale, more often identifying instruction as distinguished than observers — reports the opposite
• Most evaluators rated consistently, but about 3 percent were muted and 5 percent erratic, and about 10 percent of a sampled subset were extremely severe or lenient — related