Provide guidance to policy makers and evaluators on accounting for disengagement in their own work¶

Strategy · All strategies
Evidence · no claims cited

Description¶

The authors state they "provide guidance for policy makers and evaluators on how to account for disengagement in their own work". The guidance directs analysts using achievement test data to consider disengagement corrections, weighing that sample removal will likely induce bias while overall corrections had minimal impact on the examined metrics.

Design Implications¶

Context¶

Requirements¶

• Access to item response time metadata from computer-based assessments

Constraints¶

• The article notes the "promise and limitations of using achievement test metadata for related purposes"

Target Learners¶

• Policy makers and evaluators using achievement test data

Target Learning Goals¶

• Unbiased estimates of educational effectiveness from achievement test scores

Related Strategies¶

• Reconcile long-term policy goals with short-term accountability metrics by reviewing seasonal effects

Examples¶

-

Key Sources¶

• Megan Kuhfeld, James Soland. (2019). Using assessment metadata to quantify the impact of test disengagement on estimates of educational effectiveness. Journal of Research on Educational Effectiveness 13(1), 147-175. https://www.nwea.org/research/publication/using-assessment-metadata-to-quantify-the-impact-of-test-disengagement-on-estimates-of-educational-effectiveness/