Composite accountability indicators such as A–F letter grades fail to capture the cognitive and noncognitive competencies of deeper learning and college and career readiness¶

Claim · All claims
Evidence · 1 study · theoretical r2 · q2
Currency · newest study 2017 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2020: tool and model accuracy changes quickly

Subclaims¶

q2 i? State assessments used in composite indicators underrepresent deeper cognitive competencies, so summative indicators yield at most information about basic content knowledge and skills. → Adams 2017

Evidence¶

Adams 2017¶

Adams, C. M., Ford, T. G., Forsyth, P. B., Ware, J. K., Olsen, J. J., Lepine, J. A., Sr., Barnes, L. B., Khojasteh, J., & Mwavita, M. (2017). Next generation accountability: A vision for improvement under ESSA. Palo Alto, CA: Learning Policy Institute. https://learningpolicyinstitute.org/product/next-generation-accountability

q2 · i? · theoretical · r2

The report's conceptual analysis of Oklahoma's A–F letter grades argues that composite indicators derived from test scores cannot capture competencies such as "critical thinking and problem solving" and collaboration. It cites a 2012 RAND study finding fewer than 2% of math items assessed deeper cognitive competencies.

"At best, a summative indicator derived from student test scores may yield information about basic content knowledge and skills, but it does not capture other cognitive and noncognitive competencies required for productive functioning in the emerging economy."

Discussion¶

Related Claims¶

• Course grades and GPA predict high school and college outcomes better than standardized test scores or coursework — related
• Focus group stakeholders across roles see statewide summative assessments as only one indicator of college readiness and want information on additional skills — possibly the same claim (merge candidate)
• Noncognitive factors have a direct positive relationship to students' school performance and future outcomes — related
• More than three of four ESSA plans (41 of 53, 77%) include a postsecondary readiness indicator as a scoring element in state accountability systems — related
• Oklahoma achievement declined rather than grew across 2012–2014 under the A–F letter grade system, with the largest drops in F schools — related
• Course grades are more predictive than standardized test scores of high school and college success — related
• Analysis of 52 higher-education assessment plans found outcome-based orientation dominant, with dynamic elements subsumed under summative scoring — a narrower finding that bears on this claim
• Basics of systems thinking and conceptual modeling skills can be imparted within less than one semester — related
• Postsecondary readiness indicators carry an average weight of 16 percent of high school summative accountability scores, ranging from 5 to 33 percent — related
• Nearly 60% of surveyed college students said their summative assessment results matched other college-readiness information not very well or very little — related
• Postsecondary readiness indicators emphasize academic measures (98% of plans) and career measures (83%) far more than military measures (27%) — related
• The CRIS framework extends existing early warning systems beyond graduation and academic measures — related
• Many state assessments incorporated more sophisticated response formats to better assess college- and career-readiness — related
• Reading score distributions within A–F letter grades overlap heavily, so letter grades misrepresent the achievement of many individual students — a narrower finding that bears on this claim
• Raw postsecondary data are useful for descriptive purposes but may not give educators substantial new information beyond high school indicators — related