Response-time-based detection of test taker disengagement¶
Research Method · All research methods
Evidence · 7 claims (7 for) · 6 studies (3 theoretical, 2 associational, 1 review),q1–q2· 0 of 6 report an effect size · 6 claims rest on one study
Description¶
The chapter describes a capability for identifying test taker disengagement using response time, which underlies its analysis of what happens when test takers disengage and how disengagement should be managed during scoring. The method detects disengaged item responses such as rapid guesses, and "research using this capability has revealed that test takers generally disengage during only a portion of their items." It functions as a measurement tool for flagging responses where response correctness no longer reflects achievement level.
Accounts¶
- Response-time-based method for identifying test taker disengagement: The chapter describes a capability for identifying test taker disengagement using response time, which underlies its analysis of what happens when test takers disengage and how disengagement should be managed during scoring. The method detects disengaged item responses such as rapid guesses, and "research using this capability has revealed that test takers generally disengage during only a portion of their items." It functions as a measurement tool for flagging responses where response correctness no longer reflects achievement level. (Wise et al. (2020))
- Response-time-based identification of test taker disengagement (cessation of measurement): The page lists an associated NWEA research output, "A cessation of measurement: Identifying test taker disengagement using response time," presented as both a book and a presentation. This frames test disengagement as identifiable through response-time evidence, positioning disengagement detection as a measurement concern accompanying the rapid-guessing brief. The page prints no further detail about the framework's content. (Steven Wise (2019))
- Item response time metadata as a basis for identifying disengaged test responses: The article rests on the idea that computer-based tests produce new forms of metadata, and that "researchers have developed and validated procedures for using item response times to identify responses to items that are likely disengaged". Rapid guessing detected through response times serves as the signal for disengagement, which the study then treats through sample removal or score adjustment. (Megan Kuhfeld (2019))
Claims¶
- Rapid guessing differs from solution behavior as a form of test-taker disengagement [+W]
- Test takers generally disengage during only a portion of their items, and these disengaged responses reflect a momentary cessation of measurement [+W]
- Rapid guesses on computer-based tests reflect a construct-irrelevant response process rather than the tested knowledge, skills, and abilities [+W]
- Including rapid guesses in scoring introduces systematic measurement errors that can seriously distort test scores [+W]
- Test-taking engagement plays an essential role in the pursuit of valid scores [+W]
- Rapid guessing is a validated indicator of disengaged item responding [+W]
- Test disengagement could undermine inferences drawn from observed achievement scores [+W]
Related Research Methods¶
-
Key Sources¶
- Wise, S., & Kuhfeld, M. (2020). A cessation of measurement: Identifying test taker disengagement using response time. In M. Margolis, & R. Feinberg (Eds.), Integrating Timing Considerations to Improve Testing Practices, Routledge. https://www.nwea.org/research/publication/a-cessation-of-measurement-identifying-test-taker-disengagement-using-response-time-book/. doi:10.4324/9781351064781-11
- Steven Wise, Megan Kuhfeld. (2019). What happens when test takers disengage? Understanding and addressing rapid guessing. NWEA Research brief. https://www.nwea.org/research/publication/what-happens-when-test-takers-disengage-understanding-and-addressing-rapid-guessing/
- Megan Kuhfeld, James Soland. (2019). Using assessment metadata to quantify the impact of test disengagement on estimates of educational effectiveness. Journal of Research on Educational Effectiveness 13(1), 147-175. https://www.nwea.org/research/publication/using-assessment-metadata-to-quantify-the-impact-of-test-disengagement-on-estimates-of-educational-effectiveness/
Page metadata
| Field | Value |
|---|---|
type |
research-method |
id |
response-time-based-detection-of-test-taker-disengagement |
aliases |
item-response-time-analysis-for-detecting-test-disengagement |
title |
Response-time-based detection of test taker disengagement |
description |
The chapter describes a capability for identifying test taker disengagement using response time, which underlies its analysis of what happens when test takers disengage and how disengagement should be managed during scoring. |
status |
review |
moved_from |
element/response-time-disengagement-identification-method, theory/cessation-of-measurement-disengagement-framework |
generated |
by process:sweep-kinds on 2026-10-10 |
trust |
unverified — no one has confirmed this page's content |
sources |
3 — wise-2020, steven-wise-2019, megan-kuhfeld-2019 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.