Performance-resolution correlation is significant in 2AFC but much smaller and mostly non-significant in free-response testing¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r2 · q3
Currency · newest study 2021 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2024: tool and model accuracy changes quickly

Subclaims¶

q3 i? Tau correlations between performance and resolution were significant for all three metrics in the 2AFC condition but much smaller in the free response condition, significant only for pairs gamma. → Vuorre 2021 q3 i? Mixed effects regression models showed a significant positive performance-by-condition interaction for all three resolution metrics, indicating a stronger correlation in 2AFC. → Vuorre 2021 (2)

Evidence¶

Vuorre 2021¶

Vuorre, M., & Metcalfe, J. (2021). Measures of relative metacognitive accuracy are confounded with task performance in tasks that permit guessing. Metacognition and Learning. https://doi.org/10.1007/s11409-020-09257-1

q3 · i? · causal · r2

In the same 100-participant within-participant online experiment, Kendall's tau correlations between proportion correct and each resolution metric were computed separately by condition. 2AFC: tau = 0.28 (p < 0.001) for all three metrics; free response: da tau = 0.12 (p = 0.090), ROC gamma 0.12 (p = 0.090), pairs gamma 0.15 (p = 0.027).

"The correlation was significant for each of the three resolution metrics in the 2AFC condition. However, it was much smaller in the free response condition, where it was only significant for the concordant-discordant pairs based gamma (Table4)."

Vuorre 2021 (2)¶

Vuorre, M., & Metcalfe, J. (2021). Measures of relative metacognitive accuracy are confounded with task performance in tasks that permit guessing. Metacognition and Learning. https://doi.org/10.1007/s11409-020-09257-1

q3 · i? · causal · r2

Mixed effects regression models predicted each resolution metric from performance, condition (free response = 0, 2AFC = 1), and their interaction. The significant positive interaction terms indicate a stronger performance-resolution correlation in the 2AFC condition, supporting the guessing-confound hypothesis.

"This prediction was supported in all three models ( da: b = 1.28 (SE = 0.62), p = 0.042; gamma (pairs): b = 0.67 (SE = 0.24), p = 0.006; gamma (ROC): b = 0.61 (SE = 0.25), p = 0.018 )."

Discussion¶

Related Claims¶

• Mathematics performance is positively correlated with metacognitive resolution in multiple-choice testing across six experiments — related
• Observed performance-resolution slopes in the mathematics data were not greater than those produced by guessing in simulation — related
• Resolution metrics are smaller in 2AFC than in free-response testing despite presumably equal knowledge — related
• Children's metacognitive judgments are not explained by baseline risk preferences or response-time cues — related
• In simulations, resolution metrics become strongly positively related to knowledge once guessing is possible, even when true resolution is constant — related