A method for identifying partial test-taking engagement is validated, with effort-moderated scores to be interpreted cautiously when partial engagement is indicated¶

Claim · All claims
Evidence · 1 study · design r? · q2
Currency · newest study 2021 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person · overdue for a recheck since 2024: tool and model accuracy changes quickly

Subclaims¶

q2 i? A described method for identifying partial test-taking engagement has validation evidence supporting its use, and effort-moderated scores should be interpreted cautiously when test events indicate partial engagement. → Wise 2021

Evidence¶

Wise 2021¶

Wise, S. L., & Kuhfeld, M. (2021). A method for identifying partial test-taking engagement. Applied Measurement in Education, 34(2), 150–161. doi:10.1080/08957347.2021.1890745

q2 · i? · design · r?

Read only through the one-paragraph description NWEA's research listing gives of this article (it appears on the listing page for Wise's 2020 workshop presentation, "Validation and applications of rapid guessing to detect test taker disengagement", which was the page cited here before). The description reports a methods paper that "provides validation evidence to support its use and interpretation" for identifying partial engagement, and advises cautious interpretation of effort-moderated scores. No method details or validation statistics are given in that description.

"This paper describes a method for identifying partial engagement and provides validation evidence to support its use and interpretation. When test events indicate the presence of partial engagement, effort-moderated scores should be interpreted cautiously."

Discussion¶

Related Claims¶

• Effort-moderated scoring estimates disengaged test takers' performance by excluding disengaged responses and scoring the remaining responses — related
• Responses used in effort-moderated scoring can sometimes reflect less-than-full, or partial, engagement, making E-M scores less trustworthy — related
• The authors provide a method for identifying partial test-taking engagement, supported by validation evidence — possibly the same claim (merge candidate)
• E-M scoring assumes the remaining responses are unaffected by disengagement, and this assumption can fail under partial engagement — a narrower finding that bears on this claim
• Test-taking engagement plays an essential role in the pursuit of valid scores — related
• In simulation, effort-moderated scoring (EM-CAT and EG-CAT) yields negligible bias and nominal 95% confidence interval coverage, unlike MLE-CAT scoring — related
• Rapid guesses on computer-based tests reflect a construct-irrelevant response process rather than the tested knowledge, skills, and abilities — related
• Rapid guessing differs from solution behavior as a form of test-taker disengagement — related
• Rapid-guessing behavior on computer-based tests signals disengaged test taking that threatens score validity — related
• Computer-based tests provide unique advantages that can be used to control construct-irrelevant factors threatening test score validity — related