E-M scoring assumes the remaining responses are unaffected by disengagement, and this assumption can fail under partial engagement¶

Claim · All claims
Evidence · 1 study · design r? · q2
Currency · newest study 2021 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person · overdue for a recheck since 2024: tool and model accuracy changes quickly

Subclaims¶

q2 i? E-M scoring rests on the assumption that the remaining responses are not affected by disengagement; when that assumption fails, scores are distorted. → Steven Wise 2021

Evidence¶

Steven Wise 2021¶

Steven Wise, Megan Kuhfeld. (2021). A method for identifying partial test-taking engagement. Applied Measurement in Education, 34(2), 150-161. https://www.nwea.org/research/publication/a-method-for-identifying-partial-test-taking-engagement/

q2 · i? · design · r?

A theoretical statement from the article's abstract identifying the core assumption of E-M scoring: "the remaining responses are not affected by disengagement". The article offers no separate statistics for this assumption in the available text; it is the premise its validation work tests.

"The scoring method, however, assumes that the remaining responses are not affected by disengagement."

Discussion¶

Related Claims¶

• Disengaged test-taking responses can distort the content representation of test items — related
• Responses used in effort-moderated scoring can sometimes reflect less-than-full, or partial, engagement, making E-M scores less trustworthy — possibly the same claim (merge candidate)
• Effort-moderated scoring estimates disengaged test takers' performance by excluding disengaged responses and scoring the remaining responses — related
• E-M scoring leaves residual score distortion, suggesting some disengaged test responses are nonrapid rather than rapid guesses — related
• Rapid-guessing behavior can distort test scores and adversely affect measurement — related
• The authors provide a method for identifying partial test-taking engagement, supported by validation evidence — related
• A method for identifying partial test-taking engagement is validated, with effort-moderated scores to be interpreted cautiously when partial engagement is indicated — a broader claim this one bears on
• Test-taking engagement plays an essential role in the pursuit of valid scores — related
• Disengaged student responses raise the issue of whether engaged responses maintain the intended content representation — related
• Current measurement models underlying achievement testing assume students respond effortfully to test items — a broader claim this one bears on
• Rapid-guessing behavior on an adaptive achievement test is inconsistent with most proposed models of unmotivated test taking except the effort-moderated model — related
• The effort-moderated IRT model's parameter estimation accuracy was studied under multiple assumption violations in the context of test-taking disengagement — related
• Rapid guessing is a validated indicator of disengaged item responding — related
• Manipulation trials were excluded from performance scoring to preserve WAIS-III validity — related