Calibration is worse and resolution better for phishing emails than for legitimate emails¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r2 · q3
Currency · newest study 2019 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? Calibration was much worse for phishing emails (M = 0.18) than for legitimate ones (M = 0.09). → Casey Inez Canfield & Baruch Fischhoff & Alex Davis 2019 q3 i? Resolution was better for phishing emails (M = 0.07) than for legitimate emails (M = 0.04). → Casey Inez Canfield & Baruch Fischhoff & Alex Davis 2019 (2)

Evidence¶

Casey Inez Canfield & Baruch Fischhoff & Alex Davis 2019¶

Casey Inez Canfield & Baruch Fischhoff & Alex Davis. (2019). Better beware: comparing metacognition for phishing and legitimate emails. Metacognition and Learning. https://doi.org/10.1007/s11409-019-09197-5

q3 · i? · causal · r2

Paired comparison within the pooled sample of 250 participants across both experiments, from the Results section. The paired t-test contrasted calibration scores computed separately for the 19 phishing and 19 legitimate emails each participant judged.

"Calibration was much worse for phishing emails ( M = 0.18) than for legitimate ones ( M =0 . 0 9 ) , p a i r e d t - t e s t ,t(249) = 6.65, p < .001."

Casey Inez Canfield & Baruch Fischhoff & Alex Davis 2019 (2)¶

Casey Inez Canfield & Baruch Fischhoff & Alex Davis. (2019). Better beware: comparing metacognition for phishing and legitimate emails. Metacognition and Learning. https://doi.org/10.1007/s11409-019-09197-5

q3 · i? · causal · r2

Same pooled paired comparison of resolution scores for phishing versus legitimate emails. The article notes all participants had poor resolution (close to 0), with no relationship between resolution for the two email types.

"Resolution was better for phishing emails (M = 0.07) than for legitimate emails ( M =0 . 0 4 ) , p a i r e d t - t e s t ,t(249) = 6.95, p < .001."

Discussion¶

Related Claims¶

• Email-detection confidence judgments show good calibration but poor resolution, with aggregate overconfidence of 18% — related
• Individual and task factors fail to predict resolution, while perceived consequences predict calibration in opposite directions for phishing and legitimate emails — related
• Participants are consistently overconfident for phishing emails but relatively well calibrated for legitimate emails except at 100% confidence — related
• Future anti-phishing interventions should measure both performance and metacognition, aiming to shift the phishing calibration curve toward the legitimate-email pattern — related
• Phishing email difficulty varies widely (15–85% correct) while mean confidence varies little (80–90%), leaving some victims unaware they were fooled — related
• Better resolution on the laboratory phishing task predicts lower odds of malicious files on participants' home computers — related