Test-taking disengagement as a threat to content-domain representation in achievement testing¶

Theory · All theories
Evidence · 2 claims (2 for) · 1 study (1 associational), q3 · 0 of 1 report an effect size · 2 claims rest on one study

Description¶

The article organizes the problem of rapid guessing within a representational framework: achievement testing requires that administered items represent a target content domain, blueprints enforce this via content constraints, and disengaged responses to some items raise the question of whether engaged responses still maintain the intended representation. Its two studies then locate the threat empirically, showing that "differential rapid guessing often resulted in test events with meaningfully distorted content representation" and that engagement differences across content categories stem primarily from item reading load.

Design Implications¶

Context¶

Requirements¶

• Test blueprints specifying content constraints against which actual content representation can be evaluated

Constraints¶

• The framework concerns rapid-guessing behavior specifically, as studied in the article's two studies.

Target Learners¶

• students taking achievement tests

Target Learning Objectives¶

• valid measurement of achievement across the intended content domain

Claims¶

• Differential Rapid Guessing Distorts Content Representation [+M]
• Engagement Differences Across Content Categories Due To Reading Load [+M]

Related Theories¶

Examples¶

• Test blueprints specifying content constraints for item selection

Key Sources¶

• Steven Wise. (2020). The impact of test-taking disengagement on item content representation. Applied Measurement in Education, 33(2), 83-94. https://doi.org/10.1080/08957347.2020.1732386