No trace format supported calibrated self-evaluation: perceived-achieved correlations were low and non-significant in all conditions¶

Claim · All claims
Evidence · 1 study · causal r2 · q3
Currency · newest study 2026 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person

Subclaims¶

q3 i? AUROC2 did not differ across conditions, and the confidence-gap (discrimination) effect size did not differ across conditions. → Fernandes 2026

Evidence¶

Fernandes 2026¶

Fernandes, D., Buschek, D., Tankelevitch, L., Kosch, T., & Welsch, R. (2026). Explaining Too Much? Understanding How Large Language Model Reasoning Traces Influence Performance and Metacognition. https://arxiv.org/abs/2605.25856 doi:10.48550/arxiv.2605.25856

q3 · i? · causal · r2

Trial-level Type-2 ROC area computed per participant from ten items did not differ across conditions (F(2,555)=0.97, p=.380); the confidence-gap ANOVA likewise showed no condition difference (F(2,555)=0.84, p=.432).

"Mean AUROC2 was nearly identical across conditions (Answer-only:𝑀= 0.586,𝑆𝐷= 0.206;Full-trace: 𝑀= 0.562,𝑆𝐷= 0.192;Summary-trace: 𝑀= 0.587,𝑆𝐷=0.202) and conditions did not differ,𝐹(2,555)=0.97,𝑝=.380,𝜂 2 =.003."

Discussion¶

Learner Variables¶

• Metacognition — outcome: instruction changes it

Related Claims¶

• Self Assessment Accuracy Is Low Without Training — the general claim this finding bears on, with the studies behind it
• Full reasoning traces impaired task performance relative to both answer-only and summary-trace conditions in AI-assisted LSAT reasoning — related
• Full reasoning traces worsened calibration bias and Brier scores relative to both other conditions — related
• Guided-LLM students showed the strongest self-assessment calibration, with the highest well-calibrated rate and strongest self-assessment–performance correlations — related
• Review-attributed evidence: user confidence in model answers exceeds model internal confidence, and explanation design narrows the gap — related
• Tell and no-agent conditions showed no significant JoL–confidence correlations, with mixed results for progress and mastery — related
• Participants substantially overestimated their performance in all trace conditions, with the largest overestimation under full traces — related
• Summary traces preserved task performance at the answer-only baseline while elevating subjective appraisal — related
• Children's self-focused confidence reasoning and other-focused accuracy reasoning show no consistent correlation across four studies when mental-state language and response biases are eliminated
• The tribute reports students tend to overestimate their sport performance, and four-level model instruction may improve calibration accuracy — related
• Reasoning-trace exposure raised trust and hedonic appeal without performance benefits, shifting subjective appraisal of the interaction — related