Disengaged test-taking responses can distort the content representation of test items¶

Claim · All claims
Evidence · 1 study · associational r? · q2
Currency · newest study 2020 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2023: tool and model accuracy changes quickly

Subclaims¶

q2 i? Disengaged responses on tests can distort item content representation, beyond their known effects on scores. → Steven Wise 2020

Evidence¶

Steven Wise 2020¶

Steven Wise. (2020). The impact of test-taking disengagement on item content representation. NWEA Research brief. https://www.nwea.org/resources/the-impact-of-test-taking-disengagement-on-item-content-representation/

q2 · i? · associational · r?

The brief's own summary of the associated research states that "New research shows how disengaged responses can also distort content representation." No effect size, sample, or method details are printed in the brief text.

"Rapid-guessing can distort test scores and adversely affect measurement. New research shows how disengaged responses can also distort content representation."

Discussion¶

Related Claims¶

• Rapid-guessing behavior can distort test scores and adversely affect measurement — related
• E-M scoring leaves residual score distortion, suggesting some disengaged test responses are nonrapid rather than rapid guesses — related
• Test-taking disengagement distorts both individual and aggregated test scores — related
• Two decades of research have produced advances in both the measurement of test-taking disengagement and understanding of its distortive impact on individual and aggregated scores — related
• Responses used in effort-moderated scoring can sometimes reflect less-than-full, or partial, engagement, making E-M scores less trustworthy — related
• E-M scoring assumes the remaining responses are unaffected by disengagement, and this assumption can fail under partial engagement — related
• Disengaged test taking materially impacts individual state summative test scores but may have relatively minor impact on score aggregations — related
• Differential rapid guessing often resulted in test events with meaningfully distorted item content representation — a narrower finding that bears on this claim
• Disengaged student responses raise the issue of whether engaged responses maintain the intended content representation — a narrower finding that bears on this claim
• Very short response times (rapid guessing) indicate disengaged test taking in both high-stakes and low-stakes testing contexts — related
• Text representations reflect learners' understandings of the real world, not only texts' content — related