Research-informed and ML-based rubrics assess students' mechanistic reasoning comparably, supporting ML techniques in rubric development¶

Claim · All claims
Evidence · 1 study · design r2 · q3
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? Research-informed and ML-based rubrics assess students' reasoning comparably, so both rubric types are valid for categorizing students' reasoning. → Martin 2025

Evidence¶

Martin 2025¶

Martin, P. P., Kranz, D., & Graulich, N. (2025). Revealing Rubric Relations: Investigating the Interdependence of a Research-Informed and a Machine Learning-Based Rubric in Assessing Student Reasoning in Chemistry. International Journal of Artificial Intelligence in Education, 35, 1465–1503. https://doi.org/10.1007/s40593-024-00440-y

q3 · i? · design · r2

The article's summary of its quantitative comparison of a research-informed rubric with a fifteen-cluster ML-based rubric via χ²-Tests, Fisher's Exact Test with simulated data, and post-hoc residual analysis. The abstract states the rubrics "assess students' reasoning comparably"; the detailed test statistics appear in the portion of the article beyond the provided text.

"Our findings indicate that research-informed and ML-based rubrics assess students' reasoning comparably. Thus, both rubric types are valid for categorizing students' reasoning, underscoring the applicability of integrating ML techniques into rubric development."

Discussion¶

Related Claims¶

• k-means and k-medoids clustering merged conceptually distinct mechanistic reasoning types, making them less effective than HDBSCAN for this data — related
• HDBSCAN clustering of BERT embeddings yielded fifteen clusters capturing the content and complexity of students' mechanistic reasoning — related
• Despite generally accurate feedback, the LLM's scoring explanations could be unpredictable and prone to logical inconsistency, failing to award points even when citing the correct rubric directive — related