Standardised patient performance shows almost perfect inter-rater (kappa 0.84) and substantial intra-rater (kappa 0.75) agreement¶

Claim · All claims
Evidence · 1 study (2 entries) · quant-synthesis r1 · q3
Currency · newest study 2021 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2024: tool and model accuracy changes quickly

Subclaims¶

q3 i? The pooled kappa for inter-rater reliability between standardised patients was 0.84 (95% CI: 0.72, 0.91) with no heterogeneity. → Bobos Pavlos 2021 q3 i? The pooled kappa for intra-rater reliability of the standardised patient was 0.75 (95% CI: 0.58, 0.86) with no heterogeneity. → Bobos Pavlos 2021 (2)

Evidence¶

Bobos Pavlos 2021¶

Bobos Pavlos, Pouliopoulou Dimitra V., Harriss Alexandra, Sadi Jackie, Rushton Alison, MacDermid Joy C. (2021). A systematic review and meta-analysis of measurement properties of objective structured clinical examinations used in physical therapy licensure and a structured review of licensure practices in countries with well-developed regulation systems. PLoS ONE. https://doi.org/10.1371/journal.pone.0255696

q3 · i? · quant-synthesis · r1

Random effects meta-analysis of kappa coefficients for inter-rater reliability between standardised patients. The pooled estimate was "0.84 (95% CI: 0.72, 0.91)" with I2 = 0%. No standardised effect size was printed for this finding.

"For the inter-rater reliability between SP, the pooled estimate of kappa Coefficient based on all the available data was 0.84 (95% CI: 0.72, 0.91) with no statistical heterogeneity, I2 = 0% (Fig 7)."

Bobos Pavlos 2021 (2)¶

Bobos Pavlos, Pouliopoulou Dimitra V., Harriss Alexandra, Sadi Jackie, Rushton Alison, MacDermid Joy C. (2021). A systematic review and meta-analysis of measurement properties of objective structured clinical examinations used in physical therapy licensure and a structured review of licensure practices in countries with well-developed regulation systems. PLoS ONE. https://doi.org/10.1371/journal.pone.0255696

q3 · i? · quant-synthesis · r?

Random effects meta-analysis of kappa coefficients for intra-rater reliability of the standardised patient's self-evaluation ratings. The pooled estimate was "0.75 (95% CI: 0.58, 0.86)" with I2 = 0%. No standardised effect size was printed for this finding.

"For the intra-rater reliability between SP, the pooled estimate of kappa Coefficient was 0.75 (95% CI: 0.58, 0.86) with no presence of statistical heterogeneity, I2 = 0% (Fig 8)."

Discussion¶

Related Claims¶

• Canada (excluding Quebec) is the only country of 17 requiring a clinical competency examination for licensure of domestic physical therapy graduates — related
• Inter-assessor agreement on OSCE scores is substantial, with a pooled intraclass correlation coefficient of 0.77 — related
• Studies of simulated patient-based CST report positive effects on trainee confidence and communication performance — related