In field studies with fixed timelines, complete data collection first but delay data analysis until Phase I review feedback arrives¶

Strategy · All strategies
Evidence · 2 claims (2 for) · 1 study (1 theoretical), q3 · 0 of 1 report an effect size · 2 claims rest on one study

Description¶

For field studies whose timelines are dictated by implementation partners, the article describes an adaptation of the registered report format: a doctoral researcher conducting a summer-melt text-messaging intervention "waited to conduct data analysis until receiving feedback on her Phase I submission." This preserved the review-before-analysis benefit when launching before editorial approval was impossible, though it risks reviewers demanding substantial methodological changes after data collection.

Design Implications¶

Context¶

Requirements¶

• A field implementation timeline that permits collecting data before analysis, and willingness to risk review outcomes after data collection

Constraints¶

• Other scenarios may exist where writing and implementing a rigid preregistration is incompatible with the negotiations required for field studies

Target Learners¶

• doctoral students and field researchers in education

Target Learning Goals¶

• publishing field experiments as registered reports despite externally dictated timelines

Claims¶

• Registered reports were rated higher than typical papers on all nineteen evaluated quality criteria by 353 peer reviewers [+M]
• In a four-wave MOOC intervention study, statistically significant first-wave findings failed to replicate in subsequent preregistered waves [+M]

Related Strategies¶

• Publish two distinct findings sections, one for prespecified analyses and one for post hoc exploratory work

Examples¶

-

Key Sources¶

• Preregistration and Registered Reports. (2021). https://osf.io/ya594