Stabilization reduced overrepresentation of 10-to-19-student groups in the extremes of score distributions for every test-based indicator, with growth indicators becoming underrepresented¶

Claim · All claims
Evidence · 1 study (2 entries) · design r2 · q2
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Stabilization reduced overrepresentation of groups of 10 to 19 students in the top and bottom 5th percentile of score distributions for every test-based indicator; for proficiency indicators the stabilized representation was close to expected. → Rosendahl 2024 q2 i? For growth indicators, stabilization was substantial enough that small groups, especially those with 10 to 19 students, were underrepresented in the extremes of the stabilized score distributions. → Rosendahl 2024 (2)

Evidence¶

Rosendahl 2024¶

Rosendahl, M., Gill, B., & Starling, J. E. (2024). Stabilizing school performance indicators in New Jersey to reduce the effect of random error (REL 2025–009). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. https://ies.ed.gov/use-work/resource-library/report/descriptive-study/stabilizing-school-performance-indicators-new-jersey-reduce-effect-random-error

q2 · i? · design · r2

Descriptive analysis of representation of 10-to-19-student groups in the top and bottom 5th percentile before and after stabilization (figure 7). For ELA growth, small groups made up over 30 percent of unstabilized extremes but about 20 percent overall, and after stabilization "less than 10 percent of the unstabilized extremes."

"Stabilization reduced overrepresentation of small student groups in the extremes for every indicator."

Rosendahl 2024 (2)¶

Rosendahl, M., Gill, B., & Starling, J. E. (2024). Stabilizing school performance indicators in New Jersey to reduce the effect of random error (REL 2025–009). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. https://ies.ed.gov/use-work/resource-library/report/descriptive-study/stabilizing-school-performance-indicators-new-jersey-reduce-effect-random-error

q2 · i? · design · r2

The same extremes analysis showed growth indicators overshooting: the authors state underrepresentation "suggests that growth indicators are too noisy" for groups of fewer than 20 students to confdently identify all truly extreme groups, though underrepresentation does not make groups invisible as size cutoffs do.

"For growth indicators, stabilization was substantial enough that small groups, especially those with 10 to 19 students, were underrepresented in the extremes of the stabilized score distributions."

Discussion¶

Related Claims¶

• Stabilization makes larger adjustments to scores from smaller student groups and to more extreme scores, as expected — related
• Stabilization changed ATSI designations more than CSI designations, with the median lowest-performing subgroup larger under stabilized indicators — related
• Stabilizing test-based indicators produced substantial changes in simulated CSI designations, with larger schools replacing smaller schools — related
• Stabilization reduced the inverse relationship between student group size and score variance for test-based indicators, especially for groups of 10 to 19 students — related