Effort-moderated scoring estimates disengaged test takers' performance by excluding disengaged responses and scoring the remaining responses¶

Claim · All claims
Evidence · 1 study · design r? · q2
Currency · newest study 2021 · checked against its sources 2026-10-09 (automated, 1 of 1 entry) · not reviewed by a person · overdue for a recheck since 2024: tool and model accuracy changes quickly

Subclaims¶

q2 i? E-M scoring is intended to estimate how well a disengaged test taker would have performed had they been fully engaged, by excluding disengaged responses from scoring and estimating performance from the remaining responses. → Steven Wise 2021

Evidence¶

Steven Wise 2021¶

Steven Wise, Megan Kuhfeld. (2021). A method for identifying partial test-taking engagement. Applied Measurement in Education, 34(2), 150-161. https://www.nwea.org/research/publication/a-method-for-identifying-partial-test-taking-engagement/

q2 · i? · design · r?

The article's abstract describes the E-M scoring mechanism: it estimates fully-engaged performance by "excluding disengaged responses from scoring" and estimating performance from the remaining responses. The available text prints no empirical statistics for this description.

"Effort-moderated (E-M) scoring is intended to estimate how well a disengaged test taker would have performed had they been fully engaged. It accomplishes this adjustment by excluding disengaged responses from scoring and estimating performance from the remaining responses."

Discussion¶

Related Claims¶

• Test takers generally disengage during only a portion of their items, and these disengaged responses reflect a momentary cessation of measurement — related
• E-M scoring assumes the remaining responses are unaffected by disengagement, and this assumption can fail under partial engagement — related
• E-M scoring leaves residual score distortion, suggesting some disengaged test responses are nonrapid rather than rapid guesses — related
• A method for identifying partial test-taking engagement is validated, with effort-moderated scores to be interpreted cautiously when partial engagement is indicated — related
• Standard effort-moderated scoring accounts for roughly one-third of score distortion due to differential disengagement on low-stakes achievement tests — related
• The authors provide a method for identifying partial test-taking engagement, supported by validation evidence — related
• In simulation, effort-moderated scoring (EM-CAT and EG-CAT) yields negligible bias and nominal 95% confidence interval coverage, unlike MLE-CAT scoring — related
• Current measurement models underlying achievement testing assume students respond effortfully to test items — related
• Rapid guesses on computer-based tests reflect a construct-irrelevant response process rather than the tested knowledge, skills, and abilities — related
• Technology-enhanced assessment items were studied as a potential means of increasing student test engagement and reducing disengagement — related
• An estimated one-third of students fail to learn because of psychosocial problems that interfere with engagement in instruction — related
• The effort-moderated IRT model's parameter estimation accuracy was studied under multiple assumption violations in the context of test-taking disengagement — related
• Manipulation trials were excluded from performance scoring to preserve WAIS-III validity — related