Publication bias is present and may inflate the overall hands-on effect size estimate¶

Claim · All claims
Evidence · 1 study (2 entries) · quant-synthesis r2 · q3
Currency · newest study 2017 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? Trim-and-fill analysis found only one missing study, leaving the funnel plot approximately symmetric, and fail-safe N analysis found 4646 null-result studies would be needed to reduce the combined significance to alpha .05. → Caglak 2017 q3 i? All selected studies reported significant positive effects in favor of the experimental group, which the author offers as a reason the overall estimate is too large. → Caglak 2017 (2)

Evidence¶

Caglak 2017¶

Caglak, S. (2017). Does hands-on science practices make an ımpact on achievement in science? A meta-analysis. Journal of Education in Science, Environment and Health (JESEH), 3(1), 69-87. https://eric.ed.gov/?id=EJ1125775

q3 · i? · quant-synthesis · r2

Publication bias assessment using funnel plots with the trim and fill method and Rosenthal's file drawer analysis. The article reports "a total of 4646 studies" would be needed, and the trim-and-fill funnel plot showed only one missing study on the left side.

"Using Rosenthal (1979)‟s file drawer analysis, we found that a total of 4646 studies that would average the null results should be added to this meta -analysis in orde r to reduce the combined significance level of the effect size to the pre-defined alpha level of .05."

Caglak 2017 (2)¶

Caglak, S. (2017). Does hands-on science practices make an ımpact on achievement in science? A meta-analysis. Journal of Education in Science, Environment and Health (JESEH), 3(1), 69-87. https://eric.ed.gov/?id=EJ1125775

q3 · i? · quant-synthesis · r2

Author's interpretation of the included studies' reporting pattern: "Negative impact or no impact was not reported in the selected studies", which the author states might explain why the overall effect size estimate is found to be too large. Many included studies were theses or dissertations.

"We observed a statistically significant and also positive effect of hands -on science practices on students‟ academic achievement in all of the studies (articles and dissertations) selected for the meta -analysis study in favor or experimental group."

Discussion¶

Related Claims¶

• Publication bias does not explain the significant positive outcomes of the 7E learning cycle in this meta-analysis — related
• Funnel plot analysis showed no publication bias among the 15 studies included in the AR meta-analysis — related
• Publication-bias checks give mixed results: Egger's test indicates funnel-plot asymmetry while Trim and Fill and Fail-Safe N indicate no bias — related
• Publication-bias checks indicate the pooled effect is unlikely to result from publication bias — related
• Publication bias in the dataset is low and unimportant: Egger's test is non-significant and the trim-and-fill adjusted effect size (ES=.76) differs from the observed value by only about .02 — related
• No publication bias is detected in the meta-analysis by rank correlation, regression, Fail-Safe N, and Trim and Fill checks — related
• Publication-bias analysis indicates 51 unpublished studies would be needed to reverse the meta-analysis findings — related
• The pooled effect of multiple intelligence practices is resistant to publication bias — related
• Small sample sizes and publication bias against non-significant findings may inflate meta-analytic effect size estimates in the chemistry teaching-method categories — related