The majority of automated performance metrics in gynecological surgery have limited validity evidence, hindering transferability outside research settings¶

Claim · All claims
Evidence · 1 study (2 entries) · quant-synthesis r2 · q2–q3
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person · recheck by 2027: tool and model accuracy changes quickly

Subclaims¶

Reading the evidence codes

q is the design tier, i the impact, n the sample, and the kind with its r the rigour judged within that kind: what each code means.

This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.

q3 i? 81.8% of the 11 identified automated performance metrics showed limited validity evidence, and no APMs were used intraoperatively. → Tesfai FM 2024 q3 i? APMs inherently avoid rater bias because performance is collected automatically, but most remain confined to simulation settings. → Tesfai FM 2024 (2)

Evidence¶

Tesfai FM 2024¶

Tesfai FM, Nagi J, Morrison I, Boal M, Olaitan A, Chandrasekaran D, Stoyanov D, Lanceley A, Francis N. (2024). Objective assessment tools in laparoscopic or robotic‐assisted gynecological surgery: A systematic review. Acta Obstet Gynecol Scand. https://doi.org/10.1111/aogs.14840

q3 · i? · quant-synthesis · r2

Systematic review synthesis of the 11 identified APMs (VR simulators such as da Vinci Surgical Simulation, LapSim, LapMentor, MIST). The review reports "the majority of APMs (81.8%) has limited validity evidence" and that this hinders transferability beyond research environments.

"However, this review showed that the majority of APMs (81.8%) has limited validity evidence. This low level of validity hinders transferability outside of the research environment."

Tesfai FM 2024 (2)¶

q2 · i? · review · r3

Review analysis of response-process validity for APMs: because metrics are recorded by the simulator, rater bias is absent. The review states APMs are "automated, hence removing rater bias"; it also notes no APMs were used intraoperatively and metrics alone cannot substitute experts' input.

"All APMs inherently demonstrate response process, as they are automated, hence removing rater bias, and theoretically having 100% reliability."

Discussion¶

A systematic review of minimally invasive gynecological surgery identified 37 objective assessment tools across 72 included studies — related
Only two of 37 objective assessment tools in minimally invasive gynecological surgery showed substantial evidence of validity — a broader claim this one bears on
Most identified assessment tools demonstrate construct or concurrent validity, but predictive validity is absent across tool categories — related
No objective assessment tool in minimally invasive gynecological surgery showed predictive validity correlating scores with patient outcomes — related
Most included studies were conducted in simulation (n=48) rather than live surgery (n=32), and metrics such as eye-tracking and EEG were disproportionately used in simulation — related

Page metadata

Field	Value
`status`	`review`
`generated`	by `process:wiki-ingest` on `2026-10-31`
`trust`	unverified — no one has confirmed this page's content

Frontmatter is the page's machine-readable record — see the Schema & Guide.