Combine objective metrics with validated subjective tools and audio-visual recording to contextualize interpretation of surgical performance data¶

Strategy · All strategies
Evidence · 1 claim (1 for) · 1 study (1 review), q2 · 0 of 1 report an effect size · 1 claim rests on one study

Description¶

The review recommends that future studies pair digital metrics with concurrent validated subjective tools such as NOTSS, and with audio-visual recording to provide context to data interpretation. It states it is "crucial therefore to combine objective and subjective measures to provide a complete picture of operative events," since metrics like gaze patterns and EDA admit conflicting interpretations (deliberate focus versus tunnel-vision; task-relevant versus task-irrelevant engagement).

Design Implications¶

Context¶

Requirements¶

• Use of a concurrent, validated subjective tool such as NOTSS to provide further evidence of the validity of objective metrics

Constraints¶

• The heterogeneity of study designs limits the ability to directly compare studies or make recommendations; no consensus exists to recommend a single objective physiological measurement of CogL

Target Learners¶

• surgeons
• surgical residents
• operating room teams

Target Learning Goals¶

• valid assessment of nontechnical skills
• cognitive load optimization
• surgical performance feedback

Affordances¶

• Surgical Sabermetrics Framework

Claims¶

• Movement and proximity metrics relate to team situation awareness: teams with higher rated SA showed less movement, and time spent near the primary surgeon predicted NOTSS scores [+W]

Related Strategies¶

• Pair convenient platform analytics with transparent, education-specific evaluation criteria rather than trusting either alone
• Validate automated video analytics and reach metric consensus before widespread adoption in surgical training
• Prioritize predictive-validity research, benchmarking of existing tools, and integration of kinematic data and AI in surgical skill assessment
• Prioritize longitudinal studies tracking trainees from simulation to real-world surgical performance
• Use performance thresholds that meaningfully measure surgeon performance alongside standardized terminology when assessing surgical learning curves

Examples¶

-

Key Sources¶

• Howie Emma E., Ambler Olivia, Gunn Eilidh G.M., Dias Roger D., Wigmore Stephen J., Skipworth Richard J.E., Yule Steven J. (2024). Surgical Sabermetrics: A Scoping Review of Technology-enhanced Assessment of Nontechnical Skills in the Operating Room. Annals of Surgery. https://doi.org/10.1097/SLA.0000000000006211