Higher linguistic quality shows no consistent association with overestimation of conceptual understanding by AI-based scoring¶
Claim · All claims
Evidence · 1 study · associationalr2·q3
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly
Evidence quality · No study behind this claim is a peer-reviewed article with a DOI: it rests on a source with no registered publication record (a web page, book or unpublished report). It is kept because a design page cites it; treat it as a lead to check, not an established finding.
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q3 i? Linguistic quality was unrelated to overestimation in nine of the eleven examined scoring approaches; significant positive associations appeared only for two ML-based approaches using the science-education-specific embedding model. → Markus S. Feser & Paul L. Tschisgale 2026
Evidence¶
Markus S. Feser & Paul L. Tschisgale 2026¶
Markus S. Feser & Paul L. Tschisgale. (2026). AI-based scoring systematically underestimates conceptual understanding of linguistically weak students' explanations in physics. Physics Education, Leibniz Institute for Science and Mathematics Education, Kiel. https://orcid.org/0000-0001-8503-0951
q3 · i? · associational · r2
Multinomial logistic regression of overestimation (Δ > 0) versus agreement for the same 116 explanations. The article reports linguistic quality "was unrelated to overestimation in ni ne of the eleven examined scoring approaches", with no effect size printed for the null associations.
"Linguistic quality was unrelated to overestimation in ni ne of the eleven examined scoring approaches (Additional file 1: Tables S2 and S3)."
Discussion¶
Related Claims¶
- Lower linguistic quality of physics explanations is associated with higher odds that AI-based scoring underestimates conceptual understanding, across all eleven examined AI-based scoring approaches — related
- Expert-assigned conceptual understanding and linguistic quality scores are moderately correlated (r = .50) in the student explanation dataset — related
- ML-based scoring approaches more often overestimated expert-assigned scores, whereas LLM-based approaches more often underestimated them — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
no-consistent-linguistic-overestimation-ai-scoring |
title |
Higher linguistic quality shows no consistent association with overestimation of conceptual understanding by AI-based scoring |
description |
Higher linguistic quality shows no consistent association with overestimation of conceptual understanding by AI-based scoring |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2026 |
recheck_by |
2029 |
generated |
by process:wiki-ingest on 2026-10-10 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — markus-s-feser-paul-l-tschisgale-2026 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.