Develop a standardised simulation assessment template incorporating both subjective and objective validation assessments, adapted per neurosurgical skill¶

Strategy · All strategies
Evidence · 5 claims (5 for) · 2 studies (2 review), q3 · 0 of 2 report an effect size · 5 claims rest on one study

Description¶

The review proposes that simulation sites adopt a shared assessment template covering face, content, and construct validity so they can "collect adequate data to validate their models whilst saving them from time-intensive work to build new validity questionnaires for every simulation model". It is motivated by the finding that most included studies reported only subjective validation (face n = 18, content n = 11) with little construct validity reporting (n = 6). The template would be adapted for the four neurosurgical skills identified in the review.

Design Implications¶

Context¶

Requirements¶

• The template must incorporate both subjective and objective validation assessments and be adapted for the various neurosurgical skills identified in the review.

Constraints¶

• The review notes there is currently a lack of consistent reporting of objective validation methods, and assessing predictive validity remains logistically impractical as it requires long-term follow-up.

Target Learners¶

• neurosurgical residents and trainees

Target Learning Goals¶

• acquisition of technical neurosurgical skills (aneurysm clipping, craniotomy/burr hole, vessel suturing, tumour resection)

Claims¶

• Most reviewed simulation studies demonstrated face and content validity, but few assessed construct or predictive validity [+W]
• Cadaver models were perceived as the most anatomically realistic and outperformed placenta models on face validity, with no content-validity difference [+W]
• Neurosurgical simulation models across 18 included studies received favourable face validity ratings for visual realism, but haptic realism was a key limitation [+W]
• Construct validity evidence shows statistically significant proficiency differences favouring junior over senior groups across modalities, though junior and senior residents did not differ on some metrics [+W]
• Microanastomosis training models discriminated competence levels: untrained groups scored significantly lower than trained groups on the NOMAT tool [+W]

Related Strategies¶

• Develop and validate a reusable, cost-effective, high-fidelity training platform and build homogeneous assessment tools for aneurysm clipping simulation

Examples¶

-

Key Sources¶

• Chawla Shreya, Devi Sharmila, Calvachi Paola, Gormley William B., Rueda-Esteban Roberto. (2022). Evaluation of simulation models in neurosurgical training according to face, content, and construct validity: a systematic review. Acta Neurochirurgica. https://doi.org/10.1007/s00701-021-05003-x