Treat a drop in scores after an assessment change as potentially a measurement artifact rather than a real achievement decline¶

Strategy · All strategies
Evidence · 1 claim (1 for) · 1 study (1 associational), q2 · 0 of 1 report an effect size · 1 claim rests on one study

Description¶

When scores fall after a new assessment is introduced, communicators and analysts should first check whether the drop reflects the instrument change before reporting an achievement decline. The article shows this in practice: many "wrongly believed that the results showed dramatic declines in student achievement", while "a closer look at the results showed that student achievement may have increased." The strategy is to perform that closer look before drawing conclusions.

Design Implications¶

Context¶

Requirements¶

• Access to the new exam results and to prior achievement data for comparison

Constraints¶

• Applied in the context of a state switching to different measuring instruments

Target Learners¶

• K-12 students whose achievement is measured by state exams

Target Learning Goals¶

• Accurate public communication of achievement trends

Claims¶

• A closer look at New York's first Common Core exam results suggests student achievement may have increased [+W]

Related Strategies¶

Examples¶

-

Key Sources¶

• John Cronin, Nate Jensen. (2014). The phantom collapse of student achievement in New York. Phi Delta Kappan 96(2), 60-66. https://www.nwea.org/research/publication/the-phantom-collapse-of-student-achievement-in-new-york/