The choice of which assessment to use as the outcome measure matters more for value-added estimates than any student or peer control-variable specification choice¶
Claim · All claims
Evidence · 1 study · associationalr?·q2
Currency · newest study 2015 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? Selecting the outcome assessment is more important than control-variable modeling choices for value-added estimates. → Matthew T. Johnson 2015
Evidence¶
Matthew T. Johnson 2015¶
Matthew T. Johnson, Stephen Lipscomb, Brian Gill. (2015). Sensitivity of Teacher Value-Added Estimates to Student and Peer Control Variables. Journal of Research on Educational Effectiveness, vol. 8, issue 1. https://www.mathematica.org/publications/journal-article-sensitivity-of-teacher-valueadded-estimates-to-student-and-peer-control-variables
q2 · i? · associational · r?
Comparative analysis of specification choices in a northern state dataset. The article states that none of the control-variable modeling choices are "as important as selecting which assessment to use as the outcome measure." No effect size is printed.
"Other modeling choices have smaller practical consequences, and none are as important as selecting which assessment to use as the outcome measure."
Discussion¶
Related Claims¶
- Teacher value-added estimates are highly correlated across model specifications that differ in student and peer control variables — related
- Replacing classroom peer characteristics with teacher-level averages and allowing demographics to influence the lagged-achievement relationship have smaller practical consequences than other specification choices — related
- The working paper examines the sensitivity and precision of teacher value-added estimates under specifications differing in student-level, peer-level, and double-lagged achievement controls — a broader claim this one bears on
- Up to 26% of teachers in the bottom value-added quintile under one specification are ranked higher under another specification — related
- Mathematica developed and estimated VAMs for use in Phase 1 of the Pennsylvania evaluation pilot — related
- Alternative value-added specifications produce substantially different teacher estimates but small changes in distribution tails — related
- The Pittsburgh VAMs incorporate course-specific assessments, attendance, and course completion alongside state assessments — related
- Teacher value-added estimates are sensitive to the test scale on which they are computed — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
vam-outcome-assessment-choice-dominates-specification-choices |
title |
The choice of which assessment to use as the outcome measure matters more for value-added estimates than any student or peer control-variable specification choice |
description |
The choice of which assessment to use as the outcome measure matters more for value-added estimates than any student or peer control-variable specification choice |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2015 |
source_checked |
2026-10-20 |
generated |
by process:wiki-ingest on 2026-10-08 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — matthew-t-johnson-2015 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.