Rapid guesses on computer-based tests reflect a construct-irrelevant response process rather than the tested knowledge, skills, and abilities¶

Claim · All claims
Evidence · 1 study · associational r? · q2
Currency · newest study 2020 · checked against its sources 2026-10-15 (automated, 1 of 1 entry) · not reviewed by a person · overdue for a recheck since 2023: tool and model accuracy changes quickly

Subclaims¶

q2 i? Rapid guesses reflect a distinctly different, construct-irrelevant response process that does not reflect what a test taker knows and can do. → Steven Wise 2020

Evidence¶

Steven Wise 2020¶

Steven Wise, Megan Kuhfeld. (2020). A cessation of measurement: Identifying test taker disengagement using response time. NWEA Research presentation, National Council on Measurement in Education 2020 virtual conference. https://www.nwea.org/research/publication/a-cessation-of-measurement-identifying-test-taker-disengagement-using-response-time/ doi:10.4324/9781351064781-11

q2 · i? · associational · r?

The authors summarize evidence from their data analyses and previous research findings on test disengagement, stating that "rapid guesses reflect a distinctly different, construct-irrelevant responses process" rather than the construct of interest.

"Collectively, this evidence shows that rapid guesses reflect a distinctly different, construct-irrelevant responses process."

Discussion¶

Related Claims¶

• One measurement article argues that low test-taking motivation biases an individual student's achievement test score downward — the general claim this finding bears on, with the studies behind it
• Computer-based test advantages can be used to manage the effects of cheating as a construct-irrelevant factor — related
• Computer-based test advantages can be used to manage the effects of disengagement as a construct-irrelevant factor — related
• Computer-based tests provide unique advantages that can be used to control construct-irrelevant factors threatening test score validity — related
• E-M scoring leaves residual score distortion, suggesting some disengaged test responses are nonrapid rather than rapid guesses — related
• Rapid guessing differs from solution behavior as a form of test-taker disengagement — related
• Rapid guessing tends to negatively distort scores and diminish validity — related
• Test takers generally disengage during only a portion of their items, and these disengaged responses reflect a momentary cessation of measurement — related
• Rapid guessing can be identified from item-level response time, and its proper interpretation depends on contextual requirements — related
• A method for identifying partial test-taking engagement is validated, with effort-moderated scores to be interpreted cautiously when partial engagement is indicated — related
• Effort-moderated scoring estimates disengaged test takers' performance by excluding disengaged responses and scoring the remaining responses — related
• Test scores measure accumulated knowledge at a moment in time, not innate student ability — related
• Computer-based testing's routine capture of item-level response times has driven increased interest in applications of response time to measuring ability and achievement — related
• Differing response styles create a construct-irrelevant source of bias that distorts fundamental inferences made from survey data — related