Detecting that students are tracked to teachers does not necessarily imply bias in teacher effectiveness estimates¶

Claim · All claims
Evidence · 1 study · theoretical r3 · q2
Currency · newest study 2015 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2018: tool and model accuracy changes quickly

Subclaims¶

q2 i? Although the falsification test can determine when students are tracked to teachers, this tracking result does not necessarily imply bias in estimates of teacher effectiveness. → Dan Goldhaber 2015

Evidence¶

Dan Goldhaber 2015¶

Dan Goldhaber, Duncan Dunbar Chaplin. (2015). Assessing the "Rothstein Falsification Test": Does It Really Show Teacher Value-Added Models Are Biased? Journal of Research on Educational Effectiveness, vol. 8, issue 1. https://www.mathematica.org/publications/assessingthe-rothstein-falsification-test-does-it-really-show-teacher-valueadded-models-are-biased doi:10.1080/19345747.2014.978059

q2 · i? · theoretical · r3

Theoretical and simulation analysis reported by the authors, showing that the test's ability to detect "when students are tracked to teachers" does not by itself establish bias in teacher effectiveness estimates. No effect size is printed.

"we also show that although the test can determine when students are tracked to teachers, this tracking result does not necessarily imply bias in estimates of teacher effectiveness."

Discussion¶

Related Claims¶

• Theoretically and in simulations, the Rothstein test often falsifies unbiased VAMs and fails to falsify biased VAMs — a broader claim this one bears on
• Tracking based on lagged student achievement is usually accounted for in value-added models, which is why the falsification test can mislead — a narrower finding that bears on this claim
• The Rothstein falsification test generates large and implausible future teacher effects even when students are randomly assigned to classrooms — related
• Including track indicators in value-added models may reduce tracking-related bias for high school reading teachers — related
• Including track indicators and classroom achievement controls simultaneously reduces precision of value-added estimates — related
• Accounting for mean and standard deviation of classroom achievement may reduce bias for middle school math teachers — related
• Alternative value-added specifications produce substantially different teacher estimates but small changes in distribution tails — related
• The Rothstein falsification test is not definitive in showing bias in VAM teacher effect estimates — possibly the same claim (merge candidate)
• The Rothstein test can reject VAMs even when students are randomly assigned conditional on the covariates in the model — related
• The Rothstein falsification test can reject value-added models even when estimated teacher effects are unbiased — related
• CDM-AIR declares four falsification conditions, including moderation nullity and context redundancy — a narrower finding that bears on this claim
• Rothstein (2010) found that standard VAMs imply implausible future-teacher effects on past student achievement — related