Reliability of state test scores is highest near cut-scores or the mean and much lower for very high- or low-performing students¶
Claim · All claims
Evidence · 1 study · reviewr2·q2
Currency · newest study 2009 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person · overdue for a recheck since 2012: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? Conditional reliability follows a downward curve as scores move away from the average or cut-scores, so reliability for very high or very low scores can be much worse. → May 2009
Evidence¶
May 2009¶
May, Henry, Irma Perez-Johnson, Joshua Haimson, Samina Sattar, and Phil Gleason (2009). Using State Tests in Education Experiments: A Discussion of the Issues (NCEE 2009-013). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. https://www.mathematica.org/publications/using-state-tests-in-education-experiments-a-discussion-of-the-issues
q2 · i? · review · r2
Psychometric discussion in the reliability section, citing Lord and Novick and Hambleton and Swaminathan. The paper notes conditional reliabilities are rarely published in technical manuals, so researchers should consult the state assessment office about whether the test is unusually hard or easy for the target population.
"For most assessments, reliability is maximized near the average score on norm-referenced tests or near performance cut-scores on criterion referenced tests, with a downward curve as scores move away from the average or cut-scores"
Discussion¶
Related Claims¶
- Multiple-choice state tests are prone to ceiling and floor effects and do not measure higher-order skills well — related
- DC estimates tend to be lowest when the cut score is set at the peak of the score distribution, where inconsistent decisions are most likely — related
- Measurement precision is strongest for students in the middle of the Spanish score distribution, with CSEM near the 3.3 RIT target — a narrower finding that bears on this claim
- The book argues that quality assessments can provide accurate and timely information to maximize student learning, in contrast to punitive testing rhetoric — related
- Percent meeting standards is an imprecise metric whose cut-score concentration produced the Era 2 misconception that Chicago taught basic skills better than high-level skills — related
- Test scores at the floor of a test's score scale contain real information about students' performance, so floor scores do not distort teacher value-added estimates — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
conditional-reliability-low-at-score-tails |
title |
Reliability of state test scores is highest near cut-scores or the mean and much lower for very high- or low-performing students |
description |
Reliability of state test scores is highest near cut-scores or the mean and much lower for very high- or low-performing students |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2009 |
source_checked |
2026-10-20 |
recheck_by |
2012 |
generated |
by process:wiki-ingest on 2026-10-08 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — may-2009 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.