Non-test-based indicators such as graduation rates are poorly suited to stabilization models assuming normal distributions, which may impose excessive adjustments¶

Claim · All claims
Evidence · 1 study (2 entries) · associational r2 · q2
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person · recheck by 2027: tool and model accuracy changes quickly

Subclaims¶

q2 i? Five-year graduation rates saturate at their upper bound and lack a clear relationship between group size and variance, so the normal-distribution stabilization model was not equipped to accommodate very high scores and may have imposed excessive adjustments, especially on small student groups. → Rosendahl 2024 q2 i? Math growth, by contrast, showed the expected funnel shape, symmetric distributions with few boundary scores, and stabilization reduced the relationship between group size and variance, likely improving reliability. → Rosendahl 2024 (2)

Evidence¶

Rosendahl 2024¶

Rosendahl, M., Gill, B., & Starling, J. E. (2024). Stabilizing school performance indicators in New Jersey to reduce the effect of random error (REL 2025–009). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. https://ies.ed.gov/use-work/resource-library/report/descriptive-study/stabilizing-school-performance-indicators-new-jersey-reduce-effect-random-error

q2 · i? · design · r2

Descriptive comparison of five-year graduation rates and math growth for economically disadvantaged students (figures 2–4): unstabilized graduation distributions saturated at 100 percent regardless of group size and lacked a clear group-size–variance relationship, violating model assumptions; similar patterns appeared across all subgroups.

"In consequence, the model was not equipped to accommodate the plausibility of very high scores and may have imposed excessive adjustments, especially on small student groups."

Rosendahl 2024 (2)¶

Rosendahl, M., Gill, B., & Starling, J. E. (2024). Stabilizing school performance indicators in New Jersey to reduce the effect of random error (REL 2025–009). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. https://ies.ed.gov/use-work/resource-library/report/descriptive-study/stabilizing-school-performance-indicators-new-jersey-reduce-effect-random-error

q2 · i? · associational · r2

Descriptive analysis of math growth distributions (figures 3–4): unstabilized scores were symmetric around a mean with few boundary scores and showed the expected funnel shape; stabilization reduced variance across all group sizes and increased uniformity of variance, indicating likely improved reliability.

"Stabilization made larger adjustments to more extreme scores from smaller student groups, reducing the relationship between student group size and score variance, as indicated by the reduced funnel shape of the stabilized"

Discussion¶

Related Claims¶

• Stabilization reduced the inverse relationship between student group size and score variance for test-based indicators, especially for groups of 10 to 19 students — related
• Stabilization makes larger adjustments to scores from smaller student groups and to more extreme scores, as expected — related
• High school civic engagement is associated with higher graduation rates for all students regardless of age at entry, race, or gender — related
• Stabilizing test-based indicators produced substantial changes in simulated CSI designations, with larger schools replacing smaller schools — related
• Gaps in the scale's sample range produce distribution anomalies, with fewer ratings in the 4 to 4.99 band than in the 3 to 3.99 band — related