The response-level confidence score correlates positively and significantly with scoring accuracy¶

Claim · All claims
Evidence · 1 study · associational r2 · q2 · i3 large
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i3 Mean confidence and predictive accuracy show a significant positive correlation (r = 0.649, p < 0.01), supporting the confidence score as a proxy for scoring quality. → Fang 2026

Evidence¶

Fang 2026¶

Fang, L., Zhang, Y., Park, J., Wang, Z., Ma, P., Zhai, X. (2026). Confidence-Aware Automated Assessment of Student-Drawn Scientific Models. https://arxiv.org/abs/2606.20264 doi:10.48550/arxiv.2606.20264

q2 · i3 · associational · r2

Correlational analysis (§4.2) of mean confidence κ(x) versus predictive accuracy across all proficiency labels on the six-item dataset. The article reports a "significant positive correlation (r= 0.649,p <0.01)", strongest alignment at the 'Beginning' level.

"We find that there is a significant positive correlation (r= 0.649,p <0.01) between the two."

Discussion¶

Related Claims¶

• Each of the 12 human raters' scores correlates significantly with AEEGS scores, with coefficients ranging from 0.604 to 0.864 — related
• Model self-confidence, weighted probabilistic scoring, and ensemble modeling each significantly improve LLM-based divergent thinking scoring, raising correlation with human judges from r = 0.781 to r = 0.823 — related