The Rothstein test can reject VAMs even when students are randomly assigned conditional on the covariates in the model¶
Claim · All claims
Evidence · 1 study · theoreticalr2·q1
Currency · newest study 2012 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2015: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q1 i? The falsification test can reject VAMs even under random assignment of students conditional on model covariates. → Dan Goldhaber 2012
Evidence¶
Dan Goldhaber 2012¶
Dan Goldhaber, Duncan Chaplin. (2012). Assessing the Rothstein Test: Does It Really Show Teacher Value-Added Models Are Biased? Working Paper 5. Seattle, WA: Center for Education Data & Research, University of Washington. https://www.mathematica.org/publications/assessing-the-rothstein-test-does-it-really-show-teacher-valueadded-models-are-biased
q1 · i? · theoretical · r2
Simulation conditions constructed by the authors in which students are "randomly assigned conditional on the covariates in the model" and the test still rejects VAMs. No effect sizes printed.
"plausible conditions under which the Rothstein falsification test rejects VAMs even when there is no bias in estimated teacher effects, and even when students are randomly assigned conditional on the covariates in the model"
Discussion¶
Related Claims¶
- Rothstein (2010) found that standard VAMs imply implausible future-teacher effects on past student achievement — related
- The Rothstein falsification test is not definitive in showing bias in VAM teacher effect estimates — a broader claim this one bears on
- The Rothstein falsification test generates large and implausible future teacher effects even when students are randomly assigned to classrooms — related
- The Rothstein falsification test can reject value-added models even when estimated teacher effects are unbiased — related
- Theoretically and in simulations, the Rothstein test often falsifies unbiased VAMs and fails to falsify biased VAMs — related
- Tracking based on lagged student achievement is usually accounted for in value-added models, which is why the falsification test can mislead — related
- Detecting that students are tracked to teachers does not necessarily imply bias in teacher effectiveness estimates — related
- Student growth modeling and teacher value-added modeling are complex due to non-random assignment, measurement error, multidimensionality, and covariates — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
rothstein-test-rejects-vams-random-assignment |
title |
The Rothstein test can reject VAMs even when students are randomly assigned conditional on the covariates in the model |
description |
The Rothstein test can reject VAMs even when students are randomly assigned conditional on the covariates in the model |
status |
review |
evidence_strength |
weak |
newest_evidence |
2012 |
recheck_by |
2015 |
generated |
by process:wiki-ingest on 2026-10-08 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — dan-goldhaber-2012 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.