Test disengagement adjustment through sample removal and rapid-guess score correction¶
Research Method · All research methods
Evidence · 7 claims (4 for, 2 mixed, 1 against) · 4 studies (4 associational),q2–q3· 0 of 4 report an effect size · 7 claims rest on one study
Description¶
The article examines two techniques for accounting for test disengagement: "removing unengaged test takers from the sample" and "adjusting test scores to remove rapidly guessed items". These operate on observed scores and samples rather than on the test itself, and are evaluated for their effect on estimates of school contributions to growth, achievement gaps, and summer learning loss.
Accounts¶
- Two techniques for accounting for test disengagement: sample removal and rapid-guess score adjustment: The article examines two techniques for accounting for test disengagement: "removing unengaged test takers from the sample" and "adjusting test scores to remove rapidly guessed items". These operate on observed scores and samples rather than on the test itself, and are evaluated for their effect on estimates of school contributions to growth, achievement gaps, and summer learning loss. (Megan Kuhfeld (2019))
Claims¶
- Accounting for test disengagement had minimal impact on estimates of school growth contributions, achievement gaps, and summer learning loss [+W]
- Removing disengaged examinees from the sample will likely induce bias in estimates of educational effectiveness [-W]
- Summer learning loss estimates changed significantly under filtering but not effort-moderated scoring [+M]
- E-M scoring leaves residual score distortion, suggesting some disengaged test responses are nonrapid rather than rapid guesses [+M]
- Rapid guessing had mostly minimal effects on estimates of school contributions to student growth in this sample [+M]
- Rapid guessing did not appear to substantively affect teacher value-added estimates under the value-added specifications used [~W]
- Distortive effects of disengagement on school means were diluted by the high proportion of students exhibiting no non-effortful behavior [~W]
Related Research Methods¶
-
Key Sources¶
- Megan Kuhfeld, James Soland. (2019). Using assessment metadata to quantify the impact of test disengagement on estimates of educational effectiveness. Journal of Research on Educational Effectiveness 13(1), 147-175. https://www.nwea.org/research/publication/using-assessment-metadata-to-quantify-the-impact-of-test-disengagement-on-estimates-of-educational-effectiveness/
Page metadata
| Field | Value |
|---|---|
type |
research-method |
id |
test-disengagement-adjustment-through-sample-removal-and-rapid-guess-score-correction |
title |
Test disengagement adjustment through sample removal and rapid-guess score correction |
description |
The article examines two techniques for accounting for test disengagement: "removing unengaged test takers from the sample" and "adjusting test scores to remove rapidly guessed items". |
status |
review |
moved_from |
element/two-disengagement-accounting-techniques |
generated |
by process:sweep-kinds on 2026-10-10 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — megan-kuhfeld-2019 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.