Publication bias analyses suggest the pooled effect may be underestimated: trim-and-fill raises the mean effect from g = 0.24 to g = 0.29, with no evidence of funnel plot asymmetry¶

Claim · All claims
Evidence · 1 study · quant-synthesis r2 · q4
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q4 i? Egger's regression test indicated no evidence of funnel plot asymmetry, but trim-and-fill indicated five missing studies on the right side, and including them would increase the mean effect to g = 0.29. → Dahl-Leonard 2024

Evidence¶

Dahl-Leonard 2024¶

Dahl-Leonard, K., Hall, C., & Peacott, D. (2024). A meta-analysis of technology-delivered literacy instruction for elementary students. Education Tech Research Dev (2024) 72:1507–1538. https://doi.org/10.1007/s11423-024-10354-0

q4 · i? · quant-synthesis · r2

Publication bias analyses (funnel plot, Egger's regression, trim-and-fill) within the meta-analysis of 53 studies. The article reports "b = 0.37, df = 51, p = .51" for Egger's test and a trim-and-fill adjusted mean of g = 0.29; no standardized effect size is printed for the bias tests themselves.

"An Egger's regression test indicated no evidence of funnel plot asymmetry (b = 0.37, df = 51, p = .51). Results of the trim and fill analysis indicated that five studies were missing from the right side of the funnel plot. Including these missing studies in the random-effects model would increase the mean effect size from g = 0.24 to g = 0.29."

Discussion¶

Related Claims¶

• The pooled effect of writing instruction for US ELL students is robust to sensitivity analyses but funnel plot asymmetry suggests possible missing positive effect sizes — related
• Funnel plot asymmetry indicates publication bias, lowering the bias-adjusted effect size to 0.642 — reports the opposite
• The included studies show signs of publication bias, with an asymmetric and dispersed funnel plot — related
• Publication-bias analyses indicate a low, negligible risk of publication bias in the synthesized effect sizes — related
• Publication bias due to small study effects was limited, and sensitivity analyses supported Winsorizing an outlying effect size — related
• Publication-bias checks give mixed results: Egger's test indicates funnel-plot asymmetry while Trim and Fill and Fail-Safe N indicate no bias — related
• Funnel plot asymmetry indicates high publication bias before propensity score matching and low publication bias after matching — related
• Funnel-plot asymmetry suggests small studies with negative or null results may be missing from the simulation-versus-clinical meta-analysis — related
• Sensitivity analyses found no evidence of publication bias in the meta-analysis — reports the opposite
• Trim-and-fill analyses suggest publication bias affected three of the four meta-analyses, indicating true mean effects may be lower than originally reported. — related
• Funnel-plot asymmetry in the TBL meta-analyses is not supported by formal tests of publication bias — reports the opposite
• Funnel plot and Egger's test show no statistical evidence of publication bias for the composite household-disadvantage selection effect — related
• No evidence of publication bias in the SDL meta-analysis — related
• Sensitivity analysis and funnel plots indicated the meta-analysis results were stable and free of significant publication bias — related
• Funnel plots suggest potential publication bias across the meta-analysis outcomes — related
• Publication bias tests provide mixed evidence but overall suggest bias is unlikely — related