Evidence of publication bias exists, but the pooled video effects are robust to it¶

Claim · All claims
Evidence · 1 study · quant-synthesis r3 · q3
Currency · newest study 2021 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? A three-parameter selection model indicated affirmative studies were more likely to be published, but nullifying the swap effect would require significant studies to be 10.48 times as likely to be published, and no plausible bias could account for the supplemental-video effect. → Noetel 2021

Evidence¶

Noetel 2021¶

Noetel, M., Griffith, S., Delaney, O., Sanders, T., Parker, P., del Pozo Cruz, B., & Lonsdale, C. (2021). Video Improves Learning in Higher Education: A Systematic Review. Review of Educational Research, 91(2), 204–236. https://doi.org/10.3102/0034654321990713

q3 · i? · quant-synthesis · r3

Publication-bias assessment via funnel plots and a three-parameter selection model across all included effect sizes. The review reports the bias pattern was significant, but "a strong publication bias would be required to nullify these effects" (10.48× for swapping; none plausible for supplemental video).

"The 3PSM likelihood ratio test indicated a pattern of effect sizes consistent with publication bias (χ2(7) = 35.50, p < 0.001), such that affirmative studies appeared more likely to be published."

Discussion¶

Related Claims¶

• Replacing existing teaching with video produces small improvements in student learning in higher education (g = 0.28) — related
• GenAI effects on externally assessed learning are highly heterogeneous and conditional on prior knowledge, scaffolding, verification, and task design, with pooled effects unstable under publication-bias correction — related
• Adding supplementary video to existing teaching produces strong learning benefits (g = 0.88) — related
• Funnel-plot and fail-safe N analyses indicate small publication bias, and one-study-removal analysis indicates stable pooled effects — related
• Publication bias is present in the mathematics–writing literature, though the pooled estimate is robust in sensitivity analyses — related
• The pooled effect of writing instruction for US ELL students is robust to sensitivity analyses but funnel plot asymmetry suggests possible missing positive effect sizes — related
• Publication bias is present in the embodied vocabulary learning literature, suggesting pooled effects may overestimate the true benefit — related
• Publication bias is detected in the cross-sectional data set, and bias-corrected estimates are highly heterogeneous — related
• Publication-bias checks indicate the pooled effect is unlikely to result from publication bias — related
• Funnel-plot asymmetry ran contrary to small-study effects because larger-sample studies reported larger effect sizes, likely reflecting longer training programs — related
• Video replacing static media (e.g., text) benefits learning more than video replacing a teacher — related
• The pooled self-efficacy–partnership correlation appears robust to the file-drawer problem — related
• The pooled gamification effect is robust across dependence assumptions, quality thresholds, and model specifications, with no evidence of publication bias — related