Non-significant moderator effect sizes show markedly lower relative visibility in the published STEM education record, with roughly 40–90% lower visibility than highly significant results¶

Claim · All claims
Evidence · 1 study · quant-synthesis r3 · q3
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? Moderator effect sizes with p-values between 0.05 and 0.50 were estimated to be only 43% as likely to appear in the published record, and those with p-values between 0.50 and 1.00 only 11% as likely, relative to highly significant results. → Bicakci 2026

Evidence¶

Bicakci 2026¶

Bicakci, M., Heller, F., Stoeger, H., & Ziegler, A. (2026). From RAMSing to Rigor: Improving Moderator Analysis in STEM Education Meta-Research. Educational Psychology Review. https://doi.org/10.1007/s10648-026-10172-1

q3 · i? · quant-synthesis · r3

Model-averaged weight-function selection analysis within RoBMA-PSMA across the 1,786 moderator effect sizes from 89 meta-analyses; the weight function showed "no down-weighting for results with p < .025 (ω = 1.00)" but progressively lower weights for larger p-values.

"moderator effect sizes with p-values between 0.05 and 0.50 were estimated to be only 43% as likely to appear in the published record, corresponding to roughly 57% lower relative visibility."

Discussion¶

Related Claims¶

• The bias-adjusted mean moderator effect in STEM education meta-analyses is g = 0.27 with substantial heterogeneity, and the published record shows overwhelming evidence of publication-bias-type selection — related