OSCEs used with physical therapy students show weak internal consistency, with a pooled Cronbach's alpha of 0.55 below the acceptable range¶

Claim · All claims
Evidence · 1 study (2 entries) · quant-synthesis r2–r3 · q3
Currency · newest study 2021 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2024: tool and model accuracy changes quickly

Subclaims¶

q3 i? The pooled Cronbach's alpha between OSCE station scores was 0.55 (95% CI: 0.41, 0.67), considered moderate and slightly below the acceptable range. → Bobos Pavlos 2021 q3 i? Heterogeneity in alpha estimates was explained by the field of the OSCE (e.g., musculoskeletal), with a significant meta-regression coefficient. → Bobos Pavlos 2021 (2)

Evidence¶

Bobos Pavlos 2021¶

Bobos Pavlos, Pouliopoulou Dimitra V., Harriss Alexandra, Sadi Jackie, Rushton Alison, MacDermid Joy C. (2021). A systematic review and meta-analysis of measurement properties of objective structured clinical examinations used in physical therapy licensure and a structured review of licensure practices in countries with well-developed regulation systems. PLoS ONE. https://doi.org/10.1371/journal.pone.0255696

q3 · i? · quant-synthesis · r2

Random effects meta-analysis of four included studies reporting Cronbach's alpha for OSCE station scores in physical therapy students. The pooled estimate was "0.55, (95% CI: 0.41, 0.67)" with moderate heterogeneity I2 = 65.5%. No standardised effect size was printed for this finding.

"Based on the available data, we found a pooled estimate of Cronbach’s alpha between OSCE station scores of 0.55, (95% CI: 0.41, 0.67) with moderate statistical heterogeneity, I2 = 65.5% (Fig 2)."

Bobos Pavlos 2021 (2)¶

Bobos Pavlos, Pouliopoulou Dimitra V., Harriss Alexandra, Sadi Jackie, Rushton Alison, MacDermid Joy C. (2021). A systematic review and meta-analysis of measurement properties of objective structured clinical examinations used in physical therapy licensure and a structured review of licensure practices in countries with well-developed regulation systems. PLoS ONE. https://doi.org/10.1371/journal.pone.0255696

q3 · i? · quant-synthesis · r3

Univariate meta-regression investigating sources of the moderate heterogeneity in alpha estimates. The field of OSCE was "statistically significant p<0.0001" with R2 = 97.85% of between-study variance explained. No standardised effect size was printed.

"The regression coefficient for the field of OSCE was statistically significant p<0.0001, (0.25, 95% CI: 0.10, 0.39) with 98% of between-study variance explained (R2 = 97.85%)."

Discussion¶

Related Claims¶

• Canada (excluding Quebec) is the only country of 17 requiring a clinical competency examination for licensure of domestic physical therapy graduates — related
• Practising physical therapists outperformed physical therapy students on a single-station simulated OSCE, supporting known-groups construct validity — related
• Cronbach's alpha values for OSCE scores vary widely across studies, ranging from 0.43 to 0.93, with most studies not reaching good internal consistency — a broader claim this one bears on
• Correlations between different OSCE station scores are weak, with a pooled Pearson correlation of 0.27 — related
• Each OSCE station score correlates strongly with the total OSCE score, with a pooled Pearson correlation of 0.71 — related