Stabilization makes larger adjustments to scores from smaller student groups and to more extreme scores, as expected¶

Claim · All claims
Evidence · 1 study · associational r2 · q2
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person · recheck by 2027: tool and model accuracy changes quickly

Subclaims¶

q2 i? Across all subgroup, indicator, and year combinations, the size of stabilization adjustments was negatively correlated with student group size (average correlation –0.28) and positively correlated with the absolute value of the unstabilized z-score (average correlation 0.52). → Rosendahl 2024

Evidence¶

Rosendahl 2024¶

Rosendahl, M., Gill, B., & Starling, J. E. (2024). Stabilizing school performance indicators in New Jersey to reduce the effect of random error (REL 2025–009). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. https://ies.ed.gov/use-work/resource-library/report/descriptive-study/stabilizing-school-performance-indicators-new-jersey-reduce-effect-random-error

q2 · i? · associational · r2

Descriptive correlational analysis of NJDOE aggregated school subgroup–indicator data (2015/16–2021/22, excluding 2019/20 and 2020/21), grouped by subgroup, indicator, and year. The article reports "average correlation –0.28" with group size and "average correlation 0.52" with absolute z-score, and about 82 percent of stabilized z-scores changed by less than 0.5 standard deviation.

"The amount of adjustment was negatively correlated with student group size (average correlation –0.28) and positively correlated with the absolute value of the z-score (average correlation 0.52) for all subgroup, indicator, and year combinations, as expected."

Discussion¶

Related Claims¶

• Non-test-based indicators such as graduation rates are poorly suited to stabilization models assuming normal distributions, which may impose excessive adjustments — related
• Stabilization reduced overrepresentation of 10-to-19-student groups in the extremes of score distributions for every test-based indicator, with growth indicators becoming underrepresented — related
• Stabilization changed ATSI designations more than CSI designations, with the median lowest-performing subgroup larger under stabilized indicators — related
• Stabilizing test-based indicators produced substantial changes in simulated CSI designations, with larger schools replacing smaller schools — related
• Stabilization reduced the inverse relationship between student group size and score variance for test-based indicators, especially for groups of 10 to 19 students — related
• Supergroup accountability broadens which schools are held responsible for small subgroups but risks recreating averages that mask group differences — related