Incorporating response time into measurement models for adaptive testing must account for acceleration and idiosyncratic within-person time-accuracy variation¶

Theory · All theories
Evidence · 4 claims (4 for) · 1 study (1 associational), q2 · 0 of 1 report an effect size · 4 claims rest on one study

Description¶

The article advances an account of how response time behavior should enter measurement models in adaptive testing. It identifies two behaviors from a large adaptive assessment data set: response acceleration, "a reduction in response time for responses that occur later in the assessment," and within-person variation in the time-accuracy association. Because these behaviors are heterogeneous and idiosyncratic, the authors argue they carry "implications for models that incorporate response time and accuracy."

Design Implications¶

Context¶

Requirements¶

• Models incorporating response time and accuracy must accommodate heterogeneous acceleration by estimated ability and inconsistent within-person time-accuracy associations, as the article states these findings have such implications.

Constraints¶

• The evidence base is a single large data set from the adaptive NWEA MAP Growth Reading Assessment; the authors note their approach "may be useful in other studies of adaptive testing data," implying generalization beyond this setting is not established.

Target Learners¶

• K-12 students taking adaptive interim reading assessments

Target Learning Objectives¶

• Accurate measurement of reading achievement in adaptive testing contexts

Claims¶

• Response Acceleration Later Adaptive Assessment [+M]
• Within Person Time Accuracy Association Varies [+M]
• Response acceleration is heterogeneous by ability: lower ability estimates are associated with larger increases in acceleration [+W]
• Response-time reductions lead to lower accuracy relative to expectation for lower ability students [+W]

Related Theories¶

• Cessation-of-measurement account of test-taking disengagement identified via response time

Examples¶

-

Key Sources¶

• Domingue, B., Kanopka, K., Staug, B., Soland, J., Kuhfeld, M., Wise, S., & Piech, C. (2021). Variation in respondent speed and its implications: Evidence from an adaptive testing scenario. Journal of Educational Measurement, 58(3), 335-363. doi:10.1111/jedm.12291