Skip to content

Methods Reproducibility framework for meta-analysis: four levels from Computational Reproducibility to Outcome Reproducibility

Research Method · All research methods
Evidence · 2 claims (2 for) · 1 study (1 theoretical), q2 · 0 of 1 report an effect size · 2 claims rest on one study

Description

The article introduces a framework targeted at facilitating the reproducibility of meta-analysis, organized in four levels building on the TOP-guidelines, the Reproducibility Enhancement Principles, and Lakens et al.'s recommendations. It addresses "numerical accuracy (Computational Reproducibility: Level 1) and numerical validity (Computational Verification: Level 2)", then Analysis Reproducibility (Level 3) and Outcome Reproducibility (Level 4), which target researchers' degrees of freedom in analytical choices and database creation. The framework specifies what original authors must provide and what independent researchers or reviewers check at each level, and is demonstrated on the TRANSFER outcome of the Xie et al. meta-analysis.

Accounts

  • Methods Reproducibility framework for meta-analysis: four levels from Computational Reproducibility to Outcome Reproducibility: The article introduces a framework targeted at facilitating the reproducibility of meta-analysis, organized in four levels building on the TOP-guidelines, the Reproducibility Enhancement Principles, and Lakens et al.'s recommendations. It addresses "numerical accuracy (Computational Reproducibility: Level 1) and numerical validity (Computational Verification: Level 2)", then Analysis Reproducibility (Level 3) and Outcome Reproducibility (Level 4), which target researchers' degrees of freedom in analytical choices and database creation. The framework specifies what original authors must provide and what independent researchers or reviewers check at each level, and is demonstrated on the TRANSFER outcome of the Xie et al. meta-analysis. (Sophia C. Weissgerber et al. (2021))
  • Meta-analyses should mandatorily publish their dataset and complete analysis code prior to publication: The article argues that open-science practices should be mandatory rather than voluntary for meta-analyses: "it should thus be mandatory (and not voluntary) to publish the data set as a separate file alongside the manuscript", and it should be mandatory prior to publication to provide the complete analysis code via a repository such as OSF. This enables independent researchers to exactly recreate reported results and supports future re-analysis as statistical tools evolve. The principle rests on the TOP-guidelines and the Reproducibility Enhancement Principles. (Sophia C. Weissgerber et al. (2021))

Claims

-

Key Sources

  • Sophia C. Weissgerber, Matthias Brunmair & Ralf Rummer. (2021). Null and Void? Errors in Meta-analysis on Perceptual Disfluency and Recommendations to Improve Meta-analytical Reproducibility. Educational Psychology Review. https://doi.org/10.1007/s10648-020-09579-1
Page metadata
Field Value
type research-method
id methods-reproducibility-framework-for-meta-analysis-four-levels-from-computational-reprodu
title Methods Reproducibility framework for meta-analysis: four levels from Computational Reproducibility to Outcome Reproducibility
description The article introduces a framework targeted at facilitating the reproducibility of meta-analysis, organized in four levels building on the TOP-guidelines, the Reproducibility Enhancement Principles, and Lakens et al.'s recommendations.
status review
generated by process:settle-candidates on 2026-10-22
trust unverified — no one has confirmed this page's content

Frontmatter is the page's machine-readable record — see the Schema & Guide.