Stabilizing test-based indicators produced substantial changes in simulated CSI designations, with larger schools replacing smaller schools¶

Claim · All claims
Evidence · 1 study (2 entries) · design r2 · q2
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person · recheck by 2027: tool and model accuracy changes quickly

Subclaims¶

q2 i? Of 88 schools designated for CSI using either unstabilized or stabilized test-based indicators, 55 (63 percent) were designated under both; 17 of 72 schools designated using unstabilized indicators (24 percent) would not be designated after stabilization, and 16 of 1,908 non-designated schools (0.84 percent) would be newly designated. → Rosendahl 2024 q2 i? Median school size was 551 for schools newly designated under stabilization and 479 for schools losing their designation, versus 475 overall, indicating larger schools replaced smaller schools after stabilization. → Rosendahl 2024 (2)

Evidence¶

Rosendahl 2024¶

Rosendahl, M., Gill, B., & Starling, J. E. (2024). Stabilizing school performance indicators in New Jersey to reduce the effect of random error (REL 2025–009). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. https://ies.ed.gov/use-work/resource-library/report/descriptive-study/stabilizing-school-performance-indicators-new-jersey-reduce-effect-random-error

q2 · i? · design · r2

Simulated 2018/19 CSI designations applying NJDOE's bottom-5th-percentile-of-Title-I-schools rule to stabilized versus unstabilized composite scores (graduation-rate and ATSI-tenure criteria not simulated). Results in table 2.

"Of the 72 schools designated for CSI using unstabilized indicators, 17 (24 percent) would not be designated if indicators were stabilized. Of the 1,908 schools not designated for CSI using unstabi -lized indicators, 16 (0.84 percent) would be designated if test-based indicators were stabilized."

Rosendahl 2024 (2)¶

Rosendahl, M., Gill, B., & Starling, J. E. (2024). Stabilizing school performance indicators in New Jersey to reduce the effect of random error (REL 2025–009). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. https://ies.ed.gov/use-work/resource-library/report/descriptive-study/stabilizing-school-performance-indicators-new-jersey-reduce-effect-random-error

q2 · i? · design · r2

Median school size analysis of CSI designation changes (figure 8); overall median school size was 475 students. The article explains composite scores for smaller schools shift away from the extremes while larger schools' scores are more stationary.

"The median size of schools that received a CSI designation using stabilized test-based indicators and that did not receive one using unstabilized test-based indicators was 551 students. The median size of schools that received a CSI designation using unstabilized test-based indicators and that did not receive one using stabilized test-based indicators was 479 students."

Discussion¶

Related Claims¶

• Stabilization changed ATSI designations more than CSI designations, with the median lowest-performing subgroup larger under stabilized indicators — related
• Stabilization makes larger adjustments to scores from smaller student groups and to more extreme scores, as expected — related
• Stabilization reduced overrepresentation of 10-to-19-student groups in the extremes of score distributions for every test-based indicator, with growth indicators becoming underrepresented — related
• Stabilization reduced the inverse relationship between student group size and score variance for test-based indicators, especially for groups of 10 to 19 students — related
• Non-test-based indicators such as graduation rates are poorly suited to stabilization models assuming normal distributions, which may impose excessive adjustments — related