Applying the reproducibility checks to TRANSFER performance, the Xie et al. (2018) results are (rather) robust, but RECALL and moderator results are doubtful¶

Claim · All claims
Evidence · 1 study · quant-synthesis r1 · q2
Currency · newest study 2021 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? TRANSFER performance results survived the authors' Levels 2–4 reproducibility checks, while identified problems raise doubts about RECALL performance and the moderator analysis. → Sophia C. Weissgerber 2021

Evidence¶

Sophia C. Weissgerber 2021¶

Sophia C. Weissgerber, Matthias Brunmair & Ralf Rummer. (2021). Null and Void? Errors in Meta-analysis on Perceptual Disfluency and Recommendations to Improve Meta-analytical Reproducibility. Educational Psychology Review. https://doi.org/10.1007/s10648-020-09579-1

q2 · i? · quant-synthesis · r1

The commentary's re-analysis, using TRANSFER as the chosen outcome variable across Levels 2–4 of its framework, found the original results robust for TRANSFER but doubtful for RECALL and the moderator analysis.

"Applying reproducibility checks to TRANSFER performance as the chosen outcome variable, we found Xie's and colleagues' results to be (rather) robust. Yet, regarding RECALL performance and the moderator analysis, the identified problems raise doubts about the credibility of the reported results."

Discussion¶

Related Claims¶

• The Xie et al. (2018) disfluency meta-analysis contained objective coding errors that reversed the reported direction of two primary studies' effects — related
• Misclassification of moderating variables in the Xie et al. (2018) meta-analysis decreases the trustworthiness of its moderator analysis — a narrower finding that bears on this claim
• Several RECALL effect sizes in the Xie et al. (2018) meta-analysis could not be reproduced, and one reported value (−0.639) is inconsistent with the primary study's data — possibly the same claim (merge candidate)
• Publication-bias assessments substantially reduce the intercept prediction, often indicating no positive transfer when the identified moderators are absent — related
• Robustness checks with fixed-effect and first-difference models reproduce the three-way interaction; continuous-time growth curve modelling does not — related