Single composite accountability indicators are susceptible to gaming and Campbell's Law corruption pressures¶

Claim · All claims
Evidence · 1 study · theoretical r2 · q2
Currency · newest study 2017 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2020: tool and model accuracy changes quickly

Subclaims¶

q2 i? Because consequences attach to outcomes rather than behaviors, composite test-based indicators reinforce both desired improvement behaviors and undesired behaviors such as teaching to the test, curriculum narrowing, and cheating, so long as the outcome is the same. → Adams 2017

Evidence¶

Adams 2017¶

Adams, C. M., Ford, T. G., Forsyth, P. B., Ware, J. K., Olsen, J. J., Lepine, J. A., Sr., Barnes, L. B., Khojasteh, J., & Mwavita, M. (2017). Next generation accountability: A vision for improvement under ESSA. Palo Alto, CA: Learning Policy Institute. https://learningpolicyinstitute.org/product/next-generation-accountability

q2 · i? · theoretical · r2

The report's argumentative analysis of composite-indicator accountability systems cites prior research on gaming and test perversion (e.g., Booher-Jennings, 2005; Forte, 2010; Ho, 2008) and quotes Campbell's warning that quantitative social indicators used for decision making become subject to corruption pressures.

"Under test-based accountability, consequences and/or rewards are attached to outcomes, not behaviors. This is problematic because both desired behaviors (e.g., efforts to improve teaching practice) and undesired behaviors (teaching to the test, narrowing of the curriculum, cheating, etc.) are reinforced equally, so long as the outcome is the same"

Discussion¶

Related Claims¶

• Sanction threats on low-performing schools can raise student test scores in the short run, but the role of gaming behavior versus genuine reform has been uncertain — related
• The study evaluates the relative importance of educational reforms and gaming behavior in generating test score gains by threatened schools — possibly the same claim (merge candidate)
• Mission statements across the U15, Russell Group, and Ivy League exhibit Booth's rhetorical corruptions, with polished rhetoric of excellence obscuring concrete commitments — related
• The 1995 Illinois governance law and test-based accountability measures put Chicago schools, including Annenberg network schools, on notice that survival depended on standardized test score targets — related
• Academic misconduct emerges primarily from systemic pressures rather than individual moral failure, and academic integrity is a collective epistemic practice institutions must sustain — a broader claim this one bears on