CATs can have important advantages over conventional tests in identifying instances when examinees exhibit low effort¶

Claim · All claims
Evidence · 1 study · review r? · q2
Currency · newest study 2014 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person

Subclaims¶

q2 i? Computerized adaptive tests are better positioned than conventional tests to detect when examinees are exhibiting low effort. → Steven Wise 2014

Evidence¶

Steven Wise 2014¶

Steven Wise. (2014). The utility of adaptive testing in addressing the problem of unmotivated examinees. Journal of Computerized Adaptive Testing, 2(1), 1–17. https://www.nwea.org/research/publication/the-utility-of-adaptive-testing-in-addressing-the-problem-of-unmotivated-examinees/

q2 · i? · review · r?

The abstract reports that the integrative review "demonstrates that they can have important advantages over conventional tests in both identifying instances when examinees are exhibiting low effort"; the full text is not supplied, so no specific studies or statistics are reported here.

"This integrative review examines the motivational benefits of computerized adaptive tests (CATs), and demonstrates that they can have important advantages over conventional tests in both identifying instances when examinees are exhibiting low effort"

Discussion¶

Related Claims¶

• CATs can effectively address the validity threat posed by unmotivated examinees in low-stakes testing — a broader claim this one bears on
• Detecting noneffortful responses requires setting response time thresholds that separate effortful from noneffortful responses — related
• Response-time threshold-setting methods for detecting noneffortful item responses were compared using reading scores from over 728,923 US students in 2,056 schools — related
• Low examinee effort is a major threat to valid uses of many test scores — related
• In simulation, effort-moderated scoring (EM-CAT and EG-CAT) yields negligible bias and nominal 95% confidence interval coverage, unlike MLE-CAT scoring — related
• An effort-guided CAT that holds the provisional achievement estimate constant after rapid guesses reduces standard errors and ameliorates severe item mis-targeting — related
• Unmotivated examinees present a major threat to the validity of scores in low-stakes testing programs — related
• CAT has the potential to be more valid and reliable than non-adaptive tests — a broader claim this one bears on
• Threshold-setting methods are compared using reading scores from over 728,923 students on a computer-adaptive test of nearly 12,000 items — related
• Effort-moderated scoring exhibits improved person fit statistics compared with traditional scoring — related
• Effort-moderated scoring yields more accurate scores than traditional scoring on an adaptive achievement test — related
• Simulations of adaptive, multistage, and fixed form tests show the adaptation measures can be used operationally to identify difficulties with adaptation — related
• A proposed effort-guided adaptive test alleviates item difficulty mistargeting caused by unmotivated test taking, as shown by simulation — related