The overall effect of PBL in the tutor-background meta-analysis is modest (g = 0.27) with large, statistically significant heterogeneity¶

Claim · All claims
Evidence · 1 study · quant-synthesis r2 · q2
Currency · newest study 2023 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Overall heterogeneity is significant (Q(352) = 1756.83, p = 0.01), Tau2 = 0.13 is large relative to the overall effect size g = 0.27, and I2 = 80.0% of observed variability is attributable to heterogeneity. → Walker 2023

Evidence¶

Walker 2023¶

Walker, A., & Leary, H. (2023). Conducting Problem-Based Learning Meta-Analysis: Complexities, Implications, and Best Practices. The Interdisciplinary Journal of Problem-based Learning. https://doi.org/10.14434/ijpbl.v17i2.35844

q2 · i? · quant-synthesis · r2

Heterogeneity assessment in the re-analysis of Leary et al. (2013) data across 353 outcomes. The article reports the overall effect as "the overall effect size (g = 0.27)" with "I2 = 80.0%" of variability attributed to heterogeneity, warranting additional analyses of study differences.

"the overall heterogeneity is statistically significant, Q(352) = 1756.83, p = 0.01. It is quite large (Tau2 = 0.13) relative to the overall effect size (g = 0.27), and a large portion of that observed variability (I2 = 80.0%) can be attributed to heterogeneity"

Studies bearing on this claim¶

Does PBL as a curriculum format raise achievement over conventional instruction, and how reliably? This table gathers the studies behind this claim and the 5 narrower claims that bear on it: 4 distinct studies (2 quant-synthesis, 1 causal, 1 theoretical), q1–q3, 2 of 4 with an effect size coded. One row per study, codes as each entry records them; no effect is averaged and no verdict is drawn. Built by scripts/synthesize_claim.py from eval/synthesis/; a row that looks wrong is fixed on the claim it comes from.

Discussion¶

Related Claims¶

• Problem-based learning shows null or negative effects on basic knowledge acquisition but positive effects on conceptual understanding and transfer relative to direct instruction — the general claim this finding bears on, with the studies behind it
• Study effect sizes are heterogeneous (I² reported as 83.197%), so a random-effects model is used for the summary effect — related
• Individual PBL outcome effect sizes in the tutor-background meta-analysis span a massive range, from g = -1.26 to g = 1.91 — related
• In a re-analysis of PBL tutor-background data, neither a meta-regression nor subgroup comparisons found that tutor expertise predicts student learning — related
• Funnel plot and Egger's test indicate publication bias in the PBL tutor-background data, and trim-and-fill suggests the overall effect is overestimated — related
• Robust variance estimation shows study of origin does not impact the final effect size in the PBL tutor-background data — related
• Effect sizes across the included studies were highly heterogeneous (I2 = 97.30%), requiring a random effects model — related
• Warning effect size shows no significant heterogeneity across topics and studies — reports the opposite
• Effect sizes across GenAI studies are highly heterogeneous, requiring random-effects pooling — a narrower finding that bears on this claim
• PBL's empirical evidence base is equivocal, attributed by the essay to implementation failures rather than design flaws — related
• Problem-based learning shows a large positive pooled effect on academic achievement (REM ES = 1.560), with heterogeneous study effects — reports the opposite
• AI-supported PBL cohorts produced sixteen student-authored peer-reviewed publications and consistently high peer evaluations — related