Use cross-method validity to select the most useful and reliable assessment tools when constructing a surgical training curriculum.¶

Strategy · All strategies
Evidence · 4 claims (4 for) · 1 study (1 review), q3 · 0 of 1 report an effect size · 4 claims rest on one study

Description¶

The review recommends the use of cross-method validity, as proposed by Hung et al., as a novel assessment method. This process "can correlate performance across different training methods (e.g., in vitro exercises, simulator performance and intraoperative performance) to determine the utility of novel performance metrics." The authors suggest this approach to address the lack of standardized assessment methods, which currently prevents novel training tools from being assessed against 'gold-standard' metrics.

Design Implications¶

Context¶

Requirements¶

• Performance data across multiple training methods (in vitro exercises, simulator performance, and intraoperative performance) must be available for correlation.

Constraints¶

• Due to the lack of standardized assessment methods, novel training tools are unable to be assessed against 'gold-standard' metrics.

Target Learners¶

• gynecology trainees learning robotic hysterectomy

Target Learning Goals¶

• valid and reliable assessment of robotic surgical skill

Claims¶

• Only two of the included studies demonstrated predictive validity of their metrics for live robotic hysterectomy performance. [+W]
• Included studies demonstrated mixed evidence of construct validity, with some metrics distinguishing skill levels but others failing to do so. [+W]
• A systematic search identified six studies of performance metrics for robotic hysterectomy, revealing a lack of standardized and validated metrics specific to the procedure. [+W]
• Virtual reality simulator performance has limited correlation with intraoperative robotic hysterectomy performance. [+W]

Related Strategies¶

• Validate automated video analytics and reach metric consensus before widespread adoption in surgical training

Examples¶

-

Key Sources¶

• Arcamo K, Murugappan S, Larkins K, Mohan H, Costello A, Pendlebury A, McNally O, McBain R. (2025). Determining the metrics of competence in robotic hysterectomy: a systematic review. Journal of Robotic Surgery. https://doi.org/10.1007/s11701-025-02471-6