Principals inflated ratings at the high end of the scale, more often identifying instruction as distinguished than observers¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2010 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2013: tool and model accuracy changes quickly

Subclaims¶

q2 i? While agreeing about unsatisfactory practice, principals were much more likely than external observers to identify instruction as distinguished, inflating ratings across all ten observable components. → Lauren Sartain 2010

Evidence¶

Lauren Sartain 2010¶

Lauren Sartain, Sara Ray Stoelinga, and Emily Krone. (2010). Findings From the First Year of the Excellence in Teaching Project in Chicago Public Schools: Rethinking Teacher Evaluation. Consortium on Chicago School Research, University of Chicago. http://ccsr.uchicago.edu/publications/Joyce_TE_yr1_finaldoc.pdf

q2 · i? · design · r2

In the matched-observation data from the year-one pilot, principals had no trouble identifying unsatisfactory teaching but "principals were much more likely than external observers to identify instruction as distinguished," inflating ratings across all ten observable components.

"principals were much more likely than external observers to identify instruction as distinguished."

Discussion¶

Related Claims¶

• Principals rated some instruction components inconsistently, rating communicating with students, using assessment in instruction, and organizing physical space lower than observers, and student engagement higher — reports the opposite
• Principals and trained external observers gave aggregate-consistent Danielson Framework ratings in matched observations — reports the opposite
• Principals were far more likely than trained external observers to rate the same instruction Distinguished rather than Proficient — possibly the same claim (merge candidate)
• Principals rated teaching practice reliably at the low and middle ends of the rating scale, and most principals' severity matched trained observers — related
• Differences between principal and observer ratings at the high end of the scale were explained largely by teachers' previous evaluation ratings — related
• Across 78 observed classrooms, 46% of teacher ratings were at the unsatisfactory or basic level, with AMPS teachers earning generally higher ratings than IDS and Renaissance 2010 teachers — related
• Actual classroom observation practice often differs from best practice, with principal-teacher perception gaps on measures and observer qualifications — related
• Most evaluators rated consistently, but about 3 percent were muted and 5 percent erratic, and about 10 percent of a sampled subset were extremely severe or lenient — related