Education journal editors adopt staged TOP-aligned transparency policies, enforced with the TOP Statement¶

Strategy · All strategies
Evidence · 2 claims (2 for) · 1 study (1 theoretical), q2 · 0 of 1 report an effect size · 2 claims rest on one study

Description¶

The article recommends that editors, publishers, and editorial committees at educational psychology journals adopt transparency policies in stages, emulating higher-rated journals in other branches of psychology. Editors can "filter policies by the topic that is easiest to engage with", modeling Level 2 mandates for materials and code sharing while adopting Level 1 disclosure for data transparency. Use of the TOP Statement, a standardized disclosure questionnaire, can simplify enforcement and complies with funder expectations such as IES's SEER standards.

Design Implications¶

Context¶

Requirements¶

• Access to the TOP Factor database of journal policies (www.topfactor.org) and the TOP Statement questionnaire as templates for policy language and enforcement

Constraints¶

• The article notes materials and code sharing has fewer complications than data sharing, so data policies may need to start at Level 1 disclosure rather than mandated transparency

Target Learners¶

• journal editors and editorial committees in educational psychology
• publishers and academic societies

Target Learning Goals¶

• increasing journal transparency and reproducibility standards
• earning recognition through TOP Factor ratings

Affordances¶

• Five Stage Culture Change Roadmap Open Science

Claims¶

• Offering Open Science Badges is associated with increased uptake of data and materials sharing and preregistration [+W]
• Most evaluated journals score low on the TOP Factor, with a mean of 3.36 and a median of 1 out of a possible 29 points [+W]

Related Strategies¶

• Editors and reviewers should reward transparency about limitations so researchers are not incentivized to overstate implications

Examples¶

-

Key Sources¶

• Mellor, D. (2021). Improving norms in research culture to incentivize transparency and rigor. Educational Psychologist, 56(2). https://orcid.org/0000-0002-3125-5888