Shrinkage does not change accountability consequences for teachers of hard-to-predict students, since extreme-rating probabilities are unaffected¶

Claim · All claims
Evidence · 1 study · causal r2 · q2
Currency · newest study 2013 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2016: tool and model accuracy changes quickly

Subclaims¶

q2 i? As a result of the null effect on extreme ratings, shrinkage does not alter the consequences that teachers of hard-to-predict students receive in the accountability system. → Herrmann 2013

Evidence¶

Herrmann 2013¶

Herrmann, M., Walsh, E., Isenberg, E., & Resch, A. (2013). Shrinkage of Value-Added Estimates and Characteristics of Students with Hard-to-Predict Achievement Levels. Cambridge, MA: Mathematica Policy Research. https://www.mathematica.org/publications/shrinkage-of-valueadded-estimates-and-characteristics-of-students-with-hardtopredict-achievement-levels

q2 · i? · causal · r2

The paper links its null finding on extreme value-added ratings to accountability outcomes, stating that teachers receiving estimates in the extremes of the distribution would, "as a result, receive consequences in the accountability system." The abstract reports no separate test of consequence probabilities.

"and, as a result, receive consequences in the accountability system."

Discussion¶

Related Claims¶

• Teachers of students with hard-to-predict achievement levels tend to have less precise value-added estimates than teachers of other students — related
• Empirical Bayes shrinkage increases the precision of value-added estimates and reduces their absolute value for teachers of hard-to-predict students — related
• Shrinkage has no statistically significant effect on the relative probability that teachers of hard-to-predict students receive extreme value-added ratings — possibly the same claim (merge candidate)
• Shrinkage had no statistically significant effect on the relative probability that teachers of hard-to-predict students received consequences — possibly the same claim (merge candidate)
• Shrinkage could differentially affect consequences for teachers of fewer students or students with hard-to-predict achievement — related
• Teachers of students with low prior achievement and who receive free lunch tend to have less precise value-added estimates — related
• Empirical Bayes shrinkage could be used to address bias in proxy pre-test value-added estimates — related
• Findings on the distribution of highest-performing teachers are robust to omitting empirical Bayes shrinkage of the value-added estimates — related