Resolution metrics are smaller in 2AFC than in free-response testing despite presumably equal knowledge¶
Claim · All claims
Evidence · 1 study · causalr?·q3
Currency · newest study 2021 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2024: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q3 i? All three resolution metrics (da, pairs gamma, ROC gamma) were greater in the free response condition than the 2AFC condition. → Vuorre 2021
Evidence¶
Vuorre 2021¶
Vuorre, M., & Metcalfe, J. (2021). Measures of relative metacognitive accuracy are confounded with task performance in tasks that permit guessing. Metacognition and Learning. https://doi.org/10.1007/s11409-020-09257-1
q3 · i? · causal · r?
Online experiment with 100 Prolific participants answering 120 general information questions in a within-participant design (2AFC vs. free response), with confidence ratings after each response. Free-response da was 1.56 [1.43, 1.67] versus 2AFC 0.74 [0.65, 0.82]; proportion correct was higher in 2AFC (0.72 vs. 0.41).
"More importantly, all three resolution metrics were greater in the free response condition. This dramatic difference is surprising insofar as participants’ knowledge across the two conditions was presumably equal."
Discussion¶
Related Claims¶
- Observed performance-resolution slopes in the mathematics data were not greater than those produced by guessing in simulation — related
- Performance-resolution correlation is significant in 2AFC but much smaller and mostly non-significant in free-response testing — related
- In simulations, resolution metrics become strongly positively related to knowledge once guessing is possible, even when true resolution is constant — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
resolution-smaller-2afc-than-free-response |
title |
Resolution metrics are smaller in 2AFC than in free-response testing despite presumably equal knowledge |
description |
Resolution metrics are smaller in 2AFC than in free-response testing despite presumably equal knowledge |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2021 |
recheck_by |
2024 |
generated |
by process:wiki-ingest on 2026-10-18 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — vuorre-2021 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.