Observed performance-resolution slopes in the mathematics data were not greater than those produced by guessing in simulation¶

Claim · All claims
Evidence · 1 study · associational r2 · q2
Currency · newest study 2021 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2024: tool and model accuracy changes quickly

Subclaims¶

q2 i? One-sample t-tests comparing data slopes to simulated slopes found only one experiment's da correlation marginally greater than the simulated relation; all others were not significant. → Vuorre 2021

Evidence¶

Vuorre 2021¶

Vuorre, M., & Metcalfe, J. (2021). Measures of relative metacognitive accuracy are confounded with task performance in tasks that permit guessing. Metacognition and Learning. https://doi.org/10.1007/s11409-020-09257-1

q2 · i? · associational · r2

Comparison of each experiment's observed performance-resolution regression slope against the simulation's slope at the experiment's mean da and guessing = 0.25, using one-sided one-sample t-tests. The authors state this analysis "while far from conclusive, indicated that the data slopes...were equal to or smaller than a relationship caused by correct guessing".

"This analysis showed that only one experiment ’s da-performance correlation was marginally greater than the simulated relation, all others were not significant (Exp 1A: p = 0.218, Exp 1B: p = 0.327, Exp 1C: p = 0.224, Exp 1D: p = 0.055, Exp 1E: p = 0.473, Exp 1F: p = 0.285)."

Discussion¶

Related Claims¶

• Lowest-confidence-trial accuracy in the mathematics data was at or above the guessing rate, supporting the guessing hypothesis — related
• Mathematics performance is positively correlated with metacognitive resolution in multiple-choice testing across six experiments — reports the opposite
• In simulations, resolution metrics become strongly positively related to knowledge once guessing is possible, even when true resolution is constant — related
• Performance-resolution correlation is significant in 2AFC but much smaller and mostly non-significant in free-response testing — related
• Resolution metrics are smaller in 2AFC than in free-response testing despite presumably equal knowledge — related
• Guessing biases calibration measures toward underconfidence, strongest for the lowest performers — related