Trim-and-fill analyses suggest publication bias affected three of the four meta-analyses, indicating true mean effects may be lower than originally reported.¶

Claim · All claims
Evidence · 1 study · quant-synthesis r3 · q3
Currency · newest study 2016 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? Publication bias analysis imputed missing unpublished studies, lowering the adjusted mean effect sizes for standardized foundational skills, standardized language/comprehension, and not-standardized language/comprehension. → Wanzek 2016

Evidence¶

Wanzek 2016¶

Wanzek, J., Vaughn, S., Scammacca, N., Gatlin, B., Walker, M. A., & Capin, P. (2016). Meta-analyses of the effects of tier 2 type reading interventions in grades K-3. Educational Psychology Review, 28, 551-576. https://eric.ed.gov/?id=ED579894

q3 · i? · quant-synthesis · r3

Trim-and-fill (Card, 2012) funnel-plot asymmetry analysis applied to each meta-analysis. For standardized foundational skills, "16 studies may be missing" and the adjusted mean effect size was 0.32; standardized language/comprehension adjusted to 0.22 and not-standardized to 0.75.

"the trim-and-fill analyses found evidence of publication bias that suggested that 16 studies may be missing. The mean effect size calculated using imputed values for the missing studies was 0.32 (95% CI = 0.21, 0.43)."

Discussion¶

Related Claims¶

• Evidence of small-study effects exists for cognition and motivation, but trim-and-fill adjustment leaves the moderate motivating effect essentially unchanged — related
• Publication-bias checks give mixed results: Egger's test indicates funnel-plot asymmetry while Trim and Fill and Fail-Safe N indicate no bias — related
• Funnel plot asymmetry indicates publication bias, lowering the bias-adjusted effect size to 0.642 — related
• The pooled effect of writing instruction for US ELL students is robust to sensitivity analyses but funnel plot asymmetry suggests possible missing positive effect sizes — related
• The included studies show signs of publication bias, with an asymmetric and dispersed funnel plot — related
• Sensitivity analyses found no evidence of publication bias in the meta-analysis — reports the opposite
• Publication bias analyses suggest the pooled effect may be underestimated: trim-and-fill raises the mean effect from g = 0.24 to g = 0.29, with no evidence of funnel plot asymmetry — related
• Funnel-plot asymmetry suggests small studies with negative or null results may be missing from the simulation-versus-clinical meta-analysis — related
• Publication-bias checks indicate the pooled effect is unlikely to result from publication bias — related
• Funnel-plot asymmetry in the TBL meta-analyses is not supported by formal tests of publication bias — reports the opposite
• Publication bias tests provide mixed evidence but overall suggest bias is unlikely — reports the opposite