The Xie et al. (2018) disfluency meta-analysis contained objective coding errors that reversed the reported direction of two primary studies' effects¶

Claim · All claims
Evidence · 1 study · theoretical r3 · q2
Currency · newest study 2021 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? In Figure 2 of the meta-analysis, Rummer et al. (2016) Experiment 2 was misreported as favoring the disfluency effect when the results indicate a small advantage of fluency. → Sophia C. Weissgerber 2021 q2 i? Weissgerber and Reinhard's (2017) delayed condition was misreported as evidence against the disfluency effect when it actually favors disfluency. → Sophia C. Weissgerber 2021

Evidence¶

Sophia C. Weissgerber 2021¶

Sophia C. Weissgerber, Matthias Brunmair & Ralf Rummer. (2021). Null and Void? Errors in Meta-analysis on Perceptual Disfluency and Recommendations to Improve Meta-analytical Reproducibility. Educational Psychology Review. https://doi.org/10.1007/s10648-020-09579-1

q2 · i? · theoretical · r3

The commentary authors' re-examination of the meta-analysis's forest plot against the primary studies found that "Experiment 2 by Rummer et al." was misreported, and that Weissgerber and Reinhard's delayed condition was likewise misreported as evidence against disfluency.

"In Figure 2 of Xie's and colleagues' meta-analysis, however, Experiment 2 by Rummer et al. ( 2016) was misreported as favoring the disfluency effect. In fact, Rummer's et al.'s( 2016) results are rather indicative of the opposite: a small advantage of fluency."

Discussion¶

Related Claims¶

• Several RECALL effect sizes in the Xie et al. (2018) meta-analysis could not be reproduced, and one reported value (−0.639) is inconsistent with the primary study's data — related
• Applying the reproducibility checks to TRANSFER performance, the Xie et al. (2018) results are (rather) robust, but RECALL and moderator results are doubtful — related
• Misclassification of moderating variables in the Xie et al. (2018) meta-analysis decreases the trustworthiness of its moderator analysis — related