Test disengagement could undermine inferences drawn from observed achievement scores¶

Claim · All claims
Evidence · 1 study · associational r? · q1
Currency · newest study 2019 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person · overdue for a recheck since 2022: tool and model accuracy changes quickly

Subclaims¶

q1 i? Student disengagement during achievement testing could undermine the inferences drawn from observed scores. → Megan Kuhfeld 2019

q1 i? Test disengagement may bias effectiveness estimates computed from observed test results. → Megan Kuhfeld 2019

Evidence¶

Megan Kuhfeld 2019¶

Megan Kuhfeld, James Soland. (2019). Using assessment metadata to quantify the impact of test disengagement on estimates of educational effectiveness. Journal of Research on Educational Effectiveness 13(1), 147-175. https://www.nwea.org/research/publication/using-assessment-metadata-to-quantify-the-impact-of-test-disengagement-on-estimates-of-educational-effectiveness/ doi:10.1080/19345747.2019.1636437

q1 · i? · associational · r?

This is the article's stated motivating premise, offered as background knowledge rather than a tested result: disengagement "could undermine the inferences drawn from observed scores". The article offers no effect size for this premise in the available text.

"Educational stakeholders have long known that students might not be fully engaged when taking an achievement test and that such disengagement could undermine the inferences drawn from observed scores."

Discussion¶

Related Claims¶

• Assessment metadata can be used to quantify the impact of test disengagement on estimates of educational effectiveness — a narrower finding that bears on this claim
• Disengaged responding biases observed scores from achievement tests and surveys in practically and statistically significant ways — a broader claim this one bears on
• Disengaged test taking in international assessment programs threatens the validity of inferences about a country's educational attainment — possibly the same claim (merge candidate)
• Test-taking engagement plays an essential role in the pursuit of valid scores — a broader claim this one bears on
• Twenty years of research on disengaged test taking in international assessment programs yields six insights with implications for assessment programs — related
• Disengaged test taking materially impacts individual state summative test scores but may have relatively minor impact on score aggregations — related
• E-M scoring partially accounts for observed RIT score differences between disengaged first tests and engaged retests — related
• Student test-taking disengagement on remotely administered adaptive interim assessments differs from disengagement on the same assessment administered in school — related
• Removing disengaged examinees from the sample will likely induce bias in estimates of educational effectiveness — related
• Filtered students had much lower effort-moderated achievement, suggesting motivation filtering may bias mean estimates — related
• Computer-based test advantages can be used to manage the effects of disengagement as a construct-irrelevant factor — related
• Content alignment between a test and its content domain represents key evidence for validating test score inferences — related
• Computer-based tests provide unique advantages that can be used to control construct-irrelevant factors threatening test score validity — related
• Differing response styles create a construct-irrelevant source of bias that distorts fundamental inferences made from survey data — related
• A test score represents a pupil's performance under given conditions, not the pupil's ability, and conditions vary with the examiner and the pupil. — related