Better resolution on the laboratory phishing task predicts lower odds of malicious files on participants' home computers¶

Claim · All claims
Evidence · 1 study (2 entries) · associational r2 · q2
Currency · newest study 2019 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2022: tool and model accuracy changes quickly

Subclaims¶

q2 i? Participants with poorer resolution were more likely to have malicious files on their computers, overall r(93) = −0.27, p = .009, and for phishing emails specifically r(93) = −0.21, p = .04. → Casey Inez Canfield & Baruch Fischhoff & Alex Davis 2019 q2 i? In logistic regressions, resolution was a weakly significant predictor of the odds of malicious files for all emails; as resolution improved, the odds of malicious files were lower. → Casey Inez Canfield & Baruch Fischhoff & Alex Davis 2019 (2)

Evidence¶

Casey Inez Canfield & Baruch Fischhoff & Alex Davis 2019¶

Casey Inez Canfield & Baruch Fischhoff & Alex Davis. (2019). Better beware: comparing metacognition for phishing and legitimate emails. Metacognition and Learning. https://doi.org/10.1007/s11409-019-09197-5

q2 · i? · associational · r2

Observational comparison in Experiment 2 linking laboratory metacognition scores to home-computer monitoring data from the Security Behavior Observatory, where malicious files were identified with VirusTotal.com aggregating over 70 anti-virus scanners.

"Most users (84/93 = 90%) had malicious files on their computer, which was more likely for those with poorer resolution, both overall, r(93) = −0.27, p = .009, and for phishing emails specifically, r(93) = −0.21, p = .04."

Casey Inez Canfield & Baruch Fischhoff & Alex Davis 2019 (2)¶

Casey Inez Canfield & Baruch Fischhoff & Alex Davis. (2019). Better beware: comparing metacognition for phishing and legitimate emails. Metacognition and Learning. https://doi.org/10.1007/s11409-019-09197-5

q2 · i? · associational · r2

Logistic regression (Model 2, Table 3) predicting presence of malicious files in Experiment 2 (N = 91), controlling for total software downloads, which was a significant predictor in all models. Models 2 and 3 for all emails had the best fit and lowest AIC values.

"For all emails, overall resolution was a weakly significant predictor of the odds of malicious files. As resolution improved (i.e. increased), the odds of malicious files were lower."

Discussion¶

Related Claims¶

• Individual and task factors fail to predict resolution, while perceived consequences predict calibration in opposite directions for phishing and legitimate emails — related
• Email-detection confidence judgments show good calibration but poor resolution, with aggregate overconfidence of 18% — related
• Future anti-phishing interventions should measure both performance and metacognition, aiming to shift the phishing calibration curve toward the legitimate-email pattern — related
• Calibration is worse and resolution better for phishing emails than for legitimate emails — related