Empirical Bayes shrinkage as a precision-based adjustment to teacher value-added estimates in accountability systems¶

Theory · All theories
Evidence · 3 claims (2 for, 1 mixed) · 1 study (1 associational), q2 · 0 of 1 report an effect size · 3 claims rest on one study

Description¶

The article describes empirical Bayes shrinkage as a procedure "common in the implementation of teacher accountability systems" that is used "to adjust teacher value-added estimates by their level of precision." Estimates based on fewer students, or on students whose achievement is hard to predict, are less precise and are therefore shrunk more. The article uses this framework to ask whether the adjustment differentially affects teachers of hard-to-predict students.

Design Implications¶

Context¶

Requirements¶

• Value-added estimates with measurable levels of precision, typically reflecting the number and predictability of a teacher's students

Constraints¶

• The article examines the procedure in the context of teacher accountability systems using value-added estimates

Target Learners¶

• K-12 students in teacher accountability systems

Target Learning Objectives¶

• Accurate estimation of teacher effectiveness for accountability decisions

Claims¶

• Teachers of students with low prior achievement and who receive free lunch tend to have less precise value-added estimates [+W]
• Shrinkage could differentially affect consequences for teachers of fewer students or students with hard-to-predict achievement [+W]
• Shrinkage had no statistically significant effect on the relative probability that teachers of hard-to-predict students received consequences [~W]

Related Theories¶

• Formula-based estimation of value-added classification error rates using ordinary least squares and empirical Bayes estimators
• Hypothesis-testing framework for computing error rates of value-added performance measurement systems

Examples¶

• Value-Added Assessment System for DC Schools and Teachers (2009–2015 project context)

Key Sources¶

• Mariesa Herrmann, Elias Walsh, Eric Isenberg. (2016). Shrinkage of Value-Added Estimates and Characteristics of Students with Hard-to-Predict Achievement Levels. Statistics and Public Policy, vol. 3, issue 1. https://www.mathematica.org/publications/ja-shrinkage-of-valueadded-estimates-and-characteristics-of-students-with-hardtopredict