A modified E-M scoring method using more liberal time thresholds accounts for two-thirds or more of score distortion, outperforming standard thresholds¶
Claim · All claims
Evidence · 1 study · designr2·q2
Currency · newest study 2020 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2023: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? A modified E-M scoring method with more liberal time thresholds performed better than standard E-M scoring, accounting for two-thirds or more of the distortion. → Steven Wise 2020
Evidence¶
Steven Wise 2020¶
Steven Wise, Megan Kuhfeld. (2020). Using retest data to evaluate and improve effort-moderated scoring. Journal of Educational Measurement. https://doi.org/10.1111/jedm.12275
q2 · i? · design · r2
Second reported finding from the same retest-data evaluation: the article states the modified method "used more liberal time thresholds" and "performed better—accounting for two‐thirds or more of the distortion." No effect size is printed.
"Second, a modified E‐M scoring method that used more liberal time thresholds performed better—accounting for two‐thirds or more of the distortion."
Discussion¶
Related Claims¶
- Standard effort-moderated scoring accounts for roughly one-third of score distortion due to differential disengagement on low-stakes achievement tests — related
- E-M scoring leaves residual score distortion, suggesting some disengaged test responses are nonrapid rather than rapid guesses — related
- E-M scoring partially accounts for observed RIT score differences between disengaged first tests and engaged retests — related
- Responses used in effort-moderated scoring can sometimes reflect less-than-full, or partial, engagement, making E-M scores less trustworthy — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
liberal-threshold-em-scoring-two-thirds-distortion |
title |
A modified E-M scoring method using more liberal time thresholds accounts for two-thirds or more of score distortion, outperforming standard thresholds |
description |
A modified E-M scoring method using more liberal time thresholds accounts for two-thirds or more of score distortion, outperforming standard thresholds |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2020 |
recheck_by |
2023 |
generated |
by process:wiki-ingest on 2026-10-08 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — steven-wise-2020 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.