Three-dimensional framework of assessment construct characteristics for ML-based science assessment: Complexity, Diversity, and Structure¶
Research Method · All research methods
Evidence · 2 claims (2 for) · 1 study (1 associational),q2· 0 of 1 report an effect size · 2 claims rest on one study
Description¶
The article adopts and empirically tests a framework abstracting three fundamental features of the assessment construct for ML-based science assessments. "Complexity" captures the cognitive processes students must engage in, from identifying provided information to evaluating multiple pieces of data. "Diversity" reflects the combinations of different cognitive demands, specifically the number of dimensions of three-dimensional science learning engaged. "Structure" reflects cognitive developmental features, operationalized as alignment to levels of a validated learning progression for scientific argumentation. The framework was used to code 19 assessment items and examine associations with ML scoring accuracy.
Accounts¶
- Three-dimensional framework of assessment construct characteristics for ML-based science assessment: Complexity, Diversity, and Structure: The article adopts and empirically tests a framework abstracting three fundamental features of the assessment construct for ML-based science assessments. "Complexity" captures the cognitive processes students must engage in, from identifying provided information to evaluating multiple pieces of data. "Diversity" reflects the combinations of different cognitive demands, specifically the number of dimensions of three-dimensional science learning engaged. "Structure" reflects cognitive developmental features, operationalized as alignment to levels of a validated learning progression for scientific argumentation. The framework was used to code 19 assessment items and examine associations with ML scoring accuracy. (Haudek et al. (2024))
Claims¶
- Higher Complexity of assessment items is associated with decreased ML scoring model performance [+M]
- Higher Structure (learning progression level) of assessment items is associated with decreased ML scoring model performance [+M]
Related Research Methods¶
-
Key Sources¶
- Haudek, K. C., & Zhai, X. (2024). Examining the Effect of Assessment Construct Characteristics on Machine Learning Scoring of Scientific Argumentation. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-023-00385-8
Page metadata
| Field | Value |
|---|---|
type |
research-method |
id |
three-dimensional-framework-of-assessment-construct-characteristics-for-ml-based-science-a |
title |
Three-dimensional framework of assessment construct characteristics for ML-based science assessment: Complexity, Diversity, and Structure |
description |
The article adopts and empirically tests a framework abstracting three fundamental features of the assessment construct for ML-based science assessments. |
status |
review |
generated |
by process:settle-candidates on 2026-10-28 |
trust |
unverified — no one has confirmed this page's content |
Frontmatter is the page's machine-readable record — see the Schema & Guide.