Cessation-of-measurement account of test-taking disengagement identified via response time¶

Theory · All theories
Evidence · 4 claims (4 for) · 2 studies (2 review), q2 · 0 of 2 report an effect size · 4 claims rest on one study

Description¶

The chapter advances an account in which disengaged test taking is understood as a momentary cessation of measurement that can be identified using response time. It holds that "the potency of an item response's information depends on the strength of a positive relationship between response correctness and achievement level," so rapid guesses that break this relationship degrade score validity. The chapter uses this account to frame both what happens when test takers disengage and how disengagement should be managed during scoring, drawing an analogy to classical test theory practice in which "authors routinely remove items that do not exhibit positive item-total correlations during item analyses."

Design Implications¶

Context¶

Requirements¶

• Response time data must be available to identify disengaged (rapid-guess) item responses

Constraints¶

Target Learners¶

• test takers on large-scale and interim assessments

Target Learning Objectives¶

• accurate measurement of what test takers know and can do

Claims¶

• Disengagement Portion Of Items Momentary Cessation [+M]
• Test-taking engagement plays an essential role in the pursuit of valid scores [+W]
• Two decades of research have produced advances in both the measurement of test-taking disengagement and understanding of its distortive impact on individual and aggregated scores [+W]
• Test-taking disengagement distorts both individual and aggregated test scores [+W]

Related Theories¶

• Incorporating response time into measurement models for adaptive testing must account for acceleration and idiosyncratic within-person time-accuracy variation
• Response-time-based identification of test taker disengagement (cessation of measurement)

Examples¶

-

Key Sources¶

• Wise, S., & Kuhfeld, M. (2020). A cessation of measurement: Identifying test taker disengagement using response time. In M. Margolis, & R. Feinberg (Eds.), Integrating Timing Considerations to Improve Testing Practices, Routledge. https://www.nwea.org/research/publication/a-cessation-of-measurement-identifying-test-taker-disengagement-using-response-time-book/. doi:10.4324/9781351064781-11