NWEA research brief on rapid guessing and test disengagement¶

Element · All elements
Evidence · 5 claims (5 for) · 6 studies (3 theoretical, 1 causal, 1 associational, 1 design), q1–q2 · 0 of 6 report an effect size · 4 claims rest on one study

Description¶

A 2019 NWEA Research brief by Steven Wise and Megan Kuhfeld titled "What happens when test takers disengage? Understanding and addressing rapid guessing." Its stated purpose is that "Research provides insight into test disengagement and how disengagement should be managed in scoring." It is tagged under the topics Measurement & scaling and School & test engagement, and is associated with the cessation-of-measurement research outputs.

Design Implications¶

Context¶

Requirements¶

• Intended for audiences concerned with measurement and test engagement, per the listed topics

Constraints¶

Target Learners¶

• K-12 students taking standardized assessments

Target Learning Goals¶

• Valid test scores unaffected by rapid-guessing disengagement

Affordances¶

• Cessation Of Measurement Disengagement Framework

Claims¶

• Rapid guessing differs from solution behavior as a form of test-taker disengagement [+W]
• Rapid guesses on computer-based tests reflect a construct-irrelevant response process rather than the tested knowledge, skills, and abilities [+W]
• Proctor notification when students disengage makes a significant impact on student test engagement [+W]
• Including rapid guesses in scoring introduces systematic measurement errors that can seriously distort test scores [+W]
• Effort-moderated scoring estimates disengaged test takers' performance by excluding disengaged responses and scoring the remaining responses [+W]

Related Elements¶

• Response-time-based method for identifying test taker disengagement

Examples¶

• Use response time to identify test taker disengagement as rapid-guessing behavior

Key Sources¶

• Steven Wise, Megan Kuhfeld. (2019). What happens when test takers disengage? Understanding and addressing rapid guessing. NWEA Research brief. https://www.nwea.org/research/publication/what-happens-when-test-takers-disengage-understanding-and-addressing-rapid-guessing/