Skip to content

Brier-score decomposition into calibration, resolution, and knowledge as metrics of detection metacognition

Research Method · All research methods
Evidence · 2 claims (2 for) · 1 study (1 causal), q3 · 0 of 1 report an effect size · 2 claims rest on one study

Description

The article uses probability judgments to reveal metacognitive states of mind, scoring them with the Brier score, which "can be decomposed into three components, calibration (or reliability), resolution, and knowledge (or uncertainty)." Calibration measures whether confidence judgments correspond to relative frequencies; resolution measures the variance of correct answers associated with different probability judgments, so high-resolution individuals "are better able to discriminate between correct and incorrect judgments by assigning them different probabilities." Judgments are binned into six confidence ranges (50–59% through 100%), with lower calibration scores and higher resolution scores indicating better metacognition; knowledge, capturing outcome uncertainty, is excluded as not an element of metacognition.

Accounts

  • Brier-score decomposition into calibration, resolution, and knowledge as metrics of detection metacognition: The article uses probability judgments to reveal metacognitive states of mind, scoring them with the Brier score, which "can be decomposed into three components, calibration (or reliability), resolution, and knowledge (or uncertainty)." Calibration measures whether confidence judgments correspond to relative frequencies; resolution measures the variance of correct answers associated with different probability judgments, so high-resolution individuals "are better able to discriminate between correct and incorrect judgments by assigning them different probabilities." Judgments are binned into six confidence ranges (50–59% through 100%), with lower calibration scores and higher resolution scores indicating better metacognition; knowledge, capturing outcome uncertainty, is excluded as not an element of metacognition. (Casey Inez Canfield et al. (2019))

Claims

-

Key Sources

  • Casey Inez Canfield & Baruch Fischhoff & Alex Davis. (2019). Better beware: comparing metacognition for phishing and legitimate emails. Metacognition and Learning. https://doi.org/10.1007/s11409-019-09197-5
Page metadata
Field Value
type research-method
id brier-score-decomposition-into-calibration-resolution-and-knowledge-as-metrics-of-detectio
title Brier-score decomposition into calibration, resolution, and knowledge as metrics of detection metacognition
description The article uses probability judgments to reveal metacognitive states of mind, scoring them with the Brier score, which "can be decomposed into three components, calibration (or reliability), resolution, and knowledge (or uncertainty)." Calibration measures whether confidence judgments correspond to
status review
generated by process:settle-candidates on 2026-10-19
trust unverified — no one has confirmed this page's content

Frontmatter is the page's machine-readable record — see the Schema & Guide.