Funnel-plot asymmetry suggests possible selection or publication bias, but p-curve analyses indicate evidential value for the pooled relationships¶

Claim · All claims
Evidence · 1 study · quant-synthesis r3 · q2
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Visual inspection of the three-level funnel plot suggests potential selection or publication bias, while right-skewed p-curves for all three bivariate relationships support evidential value. → Louise David 2024

Evidence¶

Louise David 2024¶

Louise David, Felicitas Biwer, Martine Baars, Lisette Wijnia, Fred Paas, Anique de Bruin. (2024). The Relation Between Perceived Mental Effort, Monitoring Judgments, and Learning Outcomes: A Meta-Analysis. Educational Psychology Review. https://doi.org/10.1007/s10648-024-09903-z

q2 · i? · quant-synthesis · r3

Publication-bias assessment of all 236 effect sizes via a three-level funnel plot and p-curve app analyses. The funnel plot showed asymmetry; p-curve tests for all three correlations were right-skewed (e.g., effort–monitoring continuous test Z = -19.31, p < 0.001), indicating "evidential value underlying the relationships".

"Visual inspections of the plot reveal an asymmetric shape as there are for example few studies with a larger, negative effect size and larger standard error, which indicates that there could potentially be selection or publication bias."

Discussion¶

Related Claims¶

• Publication bias analyses suggest possible small-sample bias for SCED studies, with trim-and-fill reducing the adjusted mean effect to 1.01 — related
• The pooled effect of writing instruction for US ELL students is robust to sensitivity analyses but funnel plot asymmetry suggests possible missing positive effect sizes — related
• Funnel-plot asymmetry ran contrary to small-study effects because larger-sample studies reported larger effect sizes, likely reflecting longer training programs — related
• Publication bias is present in the embodied vocabulary learning literature, suggesting pooled effects may overestimate the true benefit — related
• Publication bias could not be ruled out in the pooled teacher-student SEC estimate, though the association persisted across sensitivity analyses — related
• Publication-bias checks indicate the pooled effect is unlikely to result from publication bias — reports the opposite
• Egger's test suggests publication bias may be present, with the selection-model adjusted mean effect falling to 0.36 — related
• Publication bias evidence is mixed: marginal asymmetry for tutees but no missing studies, and no bias for tutors — related