Empirical Bayes shrinkage increases the precision of value-added estimates and reduces their absolute value for teachers of hard-to-predict students¶

Claim · All claims
Evidence · 1 study · associational r? · q2
Currency · newest study 2013 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2016: tool and model accuracy changes quickly

Subclaims¶

q2 i? Shrinkage increases the precision of value-added estimates of teachers of hard-to-predict students and reduces the absolute value of those estimates. → Herrmann 2013

Evidence¶

Herrmann 2013¶

Herrmann, M., Walsh, E., Isenberg, E., & Resch, A. (2013). Shrinkage of Value-Added Estimates and Characteristics of Students with Hard-to-Predict Achievement Levels. Cambridge, MA: Mathematica Policy Research. https://www.mathematica.org/publications/shrinkage-of-valueadded-estimates-and-characteristics-of-students-with-hardtopredict-achievement-levels

q2 · i? · associational · r?

The paper's analysis of empirical Bayes shrinkage applied to teacher value-added estimates finds that shrinkage "increases their estimates’ precision and reduces the absolute value of their value-added estimates" for teachers of hard-to-predict students. No effect size is printed in the available text.

"Shrinkage increases their estimates’ precision and reduces the absolute value of their value-added estimates."

Discussion¶

Related Claims¶

• Teachers of students with hard-to-predict achievement levels tend to have less precise value-added estimates than teachers of other students — related
• Shrinkage could differentially affect consequences for teachers of fewer students or students with hard-to-predict achievement — related
• Teachers of students with low prior achievement and who receive free lunch tend to have less precise value-added estimates — related
• Shrinkage has no statistically significant effect on the relative probability that teachers of hard-to-predict students receive extreme value-added ratings — related
• Shrinkage does not change accountability consequences for teachers of hard-to-predict students, since extreme-rating probabilities are unaffected — related
• Shrinkage had no statistically significant effect on the relative probability that teachers of hard-to-predict students received consequences — related
• The working paper examines the sensitivity and precision of teacher value-added estimates under specifications differing in student-level, peer-level, and double-lagged achievement controls — related
• A K-3 school-level growth measure was estimated and examined, with validity and precision concerns suggesting cautious use for accountability — related
• Empirical Bayes shrinkage could be used to address bias in proxy pre-test value-added estimates — related
• The Teacher Team Method and Full Roster Method produce similar estimates of teacher value added — related
• Findings on the distribution of highest-performing teachers are robust to omitting empirical Bayes shrinkage of the value-added estimates — related
• The paper compares the reliability of value-added measures of teacher performance with standards for evaluations in other fields — related
• Precision-weighted shrinkage in MAIHDA yields reliable conservative estimates for small strata and helps solve the multiple-testing problem — related