Cognitive process inference in computerized testing rarely rests on strong process data¶

Claim · All claims
Evidence · 1 study · design r1 · q1
Currency · newest study 1997 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2000: tool and model accuracy changes quickly

Subclaims¶

q1 i? The authors argue that inferences about students' cognitive processing during computerized tests are rarely based on strong process data, motivating a proposed process analyzer API. → Chung 1997

Evidence¶

Chung 1997¶

Chung, G. K. W. K., Herl, H. E., Klein, D. C. D., O'Neil, H. F., Jr., & Schacter, J. (1997). Estimate of the Potential Costs and Effectiveness of Scaling Up CRESST Assessment Software. https://eric.ed.gov/?id=ED418102

q1 · i? · design · r1

This is the authors' interpretive rationale in the Process Analyzer API specification, arguing that conclusions about cognitive processing are typically based on self-reported data or data aggregated over the task. No empirical result is reported here.

"We know very little about what students actually do while they take tests in computerized environments. The inferences we draw about cognitive processing are rarely based on strong process data."

Discussion¶

Related Claims¶

• Immediate knowledge of results during computerized achievement testing produces essentially no systematic differences in achievement estimates or response dimensionality — related
• Algorithmically inferred measurement functions may intensify, rather than resolve, the problem of nomic measurement in analyzing student learning — related
• Content alignment between a test and its content domain represents key evidence for validating test score inferences — related