Provide full feedback via injected test emails in anti-phishing training, since embedded training only feedbacks failures¶

Strategy · All strategies
Evidence · 2 claims (2 for) · 1 study (1 associational), q3 · 0 of 1 report an effect size · 2 claims rest on one study

Description¶

The article recommends feedback-based interventions modeled on vigilance research, where "signal injection and performance feedback" improved detection in sonar watchstanding, baggage screening, and medical diagnosis by artificially injecting tests throughout normal task performance and giving feedback on success. Popular workplace embedded training (e.g. Wombat Security, PhishMe) sends artificial phishing emails and tells employees when they fall for them, increasing subsequent detection performance, but "it provides only partial feedback. Individuals who never click on the artificial phishing emails never receive notice that they made the right choice." The article proposes testing full feedback to support self-regulated learning, and notes error management training, which emphasizes learning from errors via emotional self-regulation and metacognition, as a parallel approach.

Design Implications¶

Context¶

Requirements¶

• Injected test emails delivered during normal email performance, with feedback on whether participants succeeded

Constraints¶

• The article states more research is needed to measure an effect of such an intervention on metacognition
• Existing embedded training provides only partial feedback to those who click

Target Learners¶

• K-12 students
• university students
• workplace employees

Target Learning Goals¶

• phishing detection
• metacognitive calibration and resolution for security judgments
• self-regulated learning from feedback

Affordances¶

• Brier Decomposition Metacognition Metrics

Claims¶

• Participants are consistently overconfident for phishing emails but relatively well calibrated for legitimate emails except at 100% confidence [+W]
• Phishing email difficulty varies widely (15–85% correct) while mean confidence varies little (80–90%), leaving some victims unaware they were fooled [+W]

Related Strategies¶

Examples¶

-

Key Sources¶

• Casey Inez Canfield & Baruch Fischhoff & Alex Davis. (2019). Better beware: comparing metacognition for phishing and legitimate emails. Metacognition and Learning. https://doi.org/10.1007/s11409-019-09197-5