State tests under NCLB measured little higher-order thinking, with only 2 percent of math items qualifying¶

Claim · All claims
Evidence · 1 study · review r2 · q2
Currency · newest study 2017 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Only 2 percent of math items and about 20 percent of English language arts items on NCLB-era state tests represented higher-order thinking skills, which the RAND evaluation attributed largely to the limits of multiple-choice testing. → Darling-Hammond 2017

Evidence¶

Darling-Hammond 2017¶

Darling-Hammond, L. (2017). Developing and Measuring Higher Order Skills: Models for State Performance Assessment Systems. Council of Chief State School Officers. https://eric.ed.gov/?id=ED606777

q2 · i? · review · r2

The paper reports a RAND Corporation evaluation of state test item depth under NCLB, finding "only 2 percent of math items" and about 20 percent of ELA items represented higher-order thinking, with multiple-choice limits cited as a major reason. No effect size is printed.

"When the RAND Corporation evaluated the depth of knowledge represented in state tests under NCLB, for example, they found that only 2 percent of math items and only about 20 percent of English language arts items represented higher-order thinking skills, and that the limitations imposed by multiple choice testing were a major reason for this 'dumbing down' of test content."

Discussion¶

Related Claims¶

• Multiple-choice state tests are prone to ceiling and floor effects and do not measure higher-order skills well — the general claim this finding bears on, with the studies behind it
• Significantly more TPP teachers encouraged students' higher-order thinking skills than district-supported comparison teachers — related
• ITBS mathematics is heavily weighted toward computation while ISAT contains up to three times as many items on algebra, geometry, measurement, and probability and statistics — related
• The Teacher Potential Project produced positive impacts on specific Common Core-aligned instructional practices, including engaging students with texts, supporting text evidence use, and supporting higher order thinking — related
• PARCC and Smarter Balanced assessments measure higher-order skills more fully than prior tests — related
• NCLB had negative effects on AI/AN education, including declines in Navajo students' standardized test scores of up to 50 percent over three years under scripted English-only programs — related
• The multiple-choice component of the revised ELA placement test strongly measured overall ELA performance, but few items strongly measured the three specific ELA domains — related
• SimCalc scale-up study showed significant positive impacts on curriculum-aligned conceptual items but no significant difference on state test items — related