Teachers rated classroom observation scores as the most accurate REACH measure of teaching effectiveness and value-added measures as the least accurate¶

Claim · All claims
Evidence · 1 study · associational r? · q2
Currency · newest study 2020 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2023: tool and model accuracy changes quickly

Subclaims¶

q2 i? Of the REACH elements (classroom observations, performance tasks, and value-added measures), teachers and administrators felt observation scores most accurately captured teacher effectiveness, while VAMs were rated least accurate. → Sartain 2020

Evidence¶

Sartain 2020¶

Sartain, L., Zou, A., Gutiérrez, V., Shyjka, A., Hinton, E., Brown, E.R., & Easton, J.Q. (2020). Teacher Evaluation in CPS: Perceptions of REACH Implementation, Five Years In. Chicago, IL: University of Chicago Consortium on School Research. https://consortium.uchicago.edu

q2 · i? · associational · r?

Survey of approximately 12,000 CPS teachers comparing perceived accuracy of REACH components. Teachers judged "observation feedback scores were the most accurate at representing their teaching effectiveness" and VAMs the least accurate; about 70 percent still said VAMs were mostly or highly accurate.

"According to approximately 12,000 CPS teachers surveyed, observation feedback scores were the most accurate at representing their teaching effectiveness and value-added measures (VAMs), based on student growth on standardized tests, were the least accurate."

Discussion¶

Related Claims¶

• Teachers and administrators diverged on using evaluation ratings for personnel decisions, with 85 percent of administrators supportive — related
• Lower REACH ratings predicted teacher exit: more than half of unsatisfactory-rated teachers left CPS and low-VAM teachers moved out of tested grades/subjects — related
• Observation scores and student surveys were moderately correlated with student learning and likely captured different dimensions of teaching quality — related
• Professional practice ratings, student surveys, and value-added measures each have the potential to differentiate teacher performance in Pittsburgh Public Schools — related
• A K-3 school-level growth measure was estimated and examined, with validity and precision concerns suggesting cautious use for accountability — related
• In one quasi-experimental study of Chicago teachers, a classroom one SD higher on a quality index raised the same teacher's observation scores by 0.07 SD and student survey ratings by 0.13 SD — related
• FFT scores correlated positively with value-added measures, with domain correlations ranging from 0.19 to 0.22 and generally statistically significant — related
• The study used two sets of analyses—rating stability and the stable component's link to future contributions—to simulate each measure's predictive accuracy — related
• Teachers view classroom observations as a fair way to evaluate their performance but largely reject student growth measures as unfair — related
• Most teachers believe their REACH evaluation relies too heavily on student growth and question the fairness of the growth assessments — related
• A majority of teachers believed REACH relied too heavily on standardized tests, with special education teachers especially critical — related
• Most teachers judged their REACH evaluators fair and able to assess instruction accurately — related
• Classroom observations revealed subtle but decisive teaching differences correlated with the difference in learning outcomes — related