Use response time to identify test taker disengagement as rapid-guessing behavior¶

Strategy · All strategies
Evidence · 3 claims (3 mixed) · 2 studies (2 associational), q2–q3 · 0 of 2 report an effect size · 3 claims rest on one study

Description¶

This strategy uses item response time on computer-based tests to detect disengagement: when a test taker responds too quickly for the item to have been meaningfully considered, the response is flagged as rapid-guessing behavior. The presentation title frames the approach as "Identifying test taker disengagement using response time," and notes that on computer-based tests such item responses "can be identified as rapid-guessing behavior."

Design Implications¶

Context¶

Requirements¶

• Computer-based test administration that records item-level response times.

Constraints¶

Target Learners¶

• Test takers on computer-based achievement tests

Target Learning Goals¶

• Detecting disengaged item responses to protect score validity

Claims¶

• Rapid-guessing behavior varies by item and is significantly related to item length, item position, and presence of ancillary reading material [~W]
• Rapid-guessing behavior occurs on only about 1% of item responses in a commercially available low-stakes assessment [~W]
• The study examines whether rapid guessing over several tests is associated with background variables such as English learner status [~W]

Related Strategies¶

• Award points for non-answers (modified Traub's rule) to discourage rapid responding on low-stakes tests

Examples¶

-

Key Sources¶

• Steven Wise, Megan Kuhfeld. (2020). A cessation of measurement: Identifying test taker disengagement using response time. NWEA Research presentation, National Council on Measurement in Education 2020 virtual conference. https://www.nwea.org/research/publication/a-cessation-of-measurement-identifying-test-taker-disengagement-using-response-time/