Stabilization changed ATSI designations more than CSI designations, with the median lowest-performing subgroup larger under stabilized indicators¶

Claim · All claims
Evidence · 1 study (2 entries) · design r2 · q2
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Of 164 schools designated for ATSI using either unstabilized or stabilized test-based indicators, 95 (57 percent) were designated under both; 42 of 137 schools designated using unstabilized data (31 percent) would not be designated after stabilization, and 27 schools would be newly designated. → Rosendahl 2024 q2 i? The median size of the lowest performing subgroup in ATSI-designated schools was 294 students using stabilized test-based indicators versus 238 using unstabilized indicators, consistent with stabilization reining in error-driven outliers among small groups. → Rosendahl 2024 (2)

Evidence¶

Rosendahl 2024¶

Rosendahl, M., Gill, B., & Starling, J. E. (2024). Stabilizing school performance indicators in New Jersey to reduce the effect of random error (REL 2025–009). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. https://ies.ed.gov/use-work/resource-library/report/descriptive-study/stabilizing-school-performance-indicators-new-jersey-reduce-effect-random-error

q2 · i? · design · r2

Simulated 2018/19 ATSI designations applying New Jersey's rule (any school with a subgroup composite below the CSI threshold) to stabilized versus unstabilized test-based indicators; results in table 3.

"Of 137 schools designated using unsta -bilized data, 42 (31 percent) would not be designated if indicators were stabilized, and they would be partly replaced by a smaller group of newly designated schools. Stabilization results in larger changes to the ATSI list than to the CSI list, as expected."

Rosendahl 2024 (2)¶

Rosendahl, M., Gill, B., & Starling, J. E. (2024). Stabilizing school performance indicators in New Jersey to reduce the effect of random error (REL 2025–009). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. https://ies.ed.gov/use-work/resource-library/report/descriptive-study/stabilizing-school-performance-indicators-new-jersey-reduce-effect-random-error

q2 · i? · design · r2

Median lowest-performing-subgroup size comparison for ATSI designations under stabilized versus unstabilized test-based indicators, reported alongside table 3 for the 2018/19 school year.

"The median size of the lowest performing subgroup in schools that received an ATSI designation was 294 students when stabilized test-based indicators were used and 238 students when unstabilized test-based indicators were used."

Discussion¶

Related Claims¶

• Stabilizing test-based indicators produced substantial changes in simulated CSI designations, with larger schools replacing smaller schools — related
• Stabilization makes larger adjustments to scores from smaller student groups and to more extreme scores, as expected — related
• Stabilization reduced overrepresentation of 10-to-19-student groups in the extremes of score distributions for every test-based indicator, with growth indicators becoming underrepresented — related
• Stabilization reduced the inverse relationship between student group size and score variance for test-based indicators, especially for groups of 10 to 19 students — related