The study used two sets of analyses—rating stability and the stable component's link to future contributions—to simulate each measure's predictive accuracy¶
Claim · All claims
Evidence · 1 study · associationalr2·q2
Currency · newest study 2016 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? The study assessed the stability of each measure's ratings by examining associations between earlier- and later-year ratings, and examined the relationship between the stable part of each rating and future contributions, then simulated predictive accuracy. → Hanley Chiang 2016
Evidence¶
Hanley Chiang 2016¶
Hanley Chiang, Moira McCullough, Stephen Lipscomb, Brian Gill. (2016). Performance Measures Based on Test Scores – How Well Do They Predict Principals' Impacts? (In Focus Brief). Cambridge, MA: Mathematica Policy Research. https://www.mathematica.org/publications/performance-measures-based-on-test-scores-issue-brief
q2 · i? · associational · r2
The brief describes its two sets of analyses: rating stability across years, and the relationship between "the stable part of each principal's rating" and future contributions. Results from both fed a simulation of each measure's accuracy for predicting the following year's contributions.
"First, the study assessed the extent to which ratings from each measure are stable by examining the association between principals' ratings from earlier and later years. Second, the study examined the relationship between the stable part of each principal's rating and his or her contributions to student achievement in future years."
Discussion¶
Related Claims¶
- Average achievement and adjusted average achievement provide no information for predicting principals' contributions to student achievement in the following year — related
- Averaging principal performance measures across multiple recent years does not improve their accuracy for predicting principals' contributions in the following year — related
- Prior Knowledge Needed For Accurate Self Assessment — related
- Early ability to delay gratification predicts later academic and social outcomes, but the association is weaker and more context-dependent than originally reported. — related
- School value-added and adjusted school value-added provide, at most, a small amount of information for predicting principals' contributions in the following year — related
- Variation across schools in principal-assigned professional practice ratings suggests principals' standards may not be fully consistent across schools — related
- School value added provides very poor information for revealing principals' persistent levels of effectiveness — related
- Insight survey domain averages are more stable across years when a school keeps the same leader, suggesting principals affect these measures — related
- There is little prior evidence on the validity of principal evaluation measures, and this is the first study linking practice ratings to credible value-added measures — related
- The study assessed rating stability for 2,424 principals and predictive validity for 123 principals using Pennsylvania statewide data — possibly the same claim (merge candidate)
- School value-added and adjusted school value-added provide at most a small amount of information for predicting principals' future contributions to student achievement — related
- Teachers rated classroom observation scores as the most accurate REACH measure of teaching effectiveness and value-added measures as the least accurate — related
- Value-added estimates are likely to be noisy with the amount of data typically used in practice — related
- Component measure ratings had low to moderate positive correlations with each other, suggesting the components measure distinct dimensions of principal performance — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
two-analysis-approach-principal-measure-accuracy |
title |
The study used two sets of analyses—rating stability and the stable component's link to future contributions—to simulate each measure's predictive accuracy |
description |
The study used two sets of analyses—rating stability and the stable component's link to future contributions—to simulate each measure's predictive accuracy |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2016 |
source_checked |
2026-10-20 |
generated |
by process:wiki-ingest on 2026-10-08 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — hanley-chiang-2016 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.