The megastudy: a massive field experiment comparing many interventions in one population on one objectively measured outcome¶

Theory · All theories
Evidence · 5 claims (3 for, 2 mixed) · 1 study (1 causal), q3 · 0 of 1 report an effect size · 5 claims rest on one study

Description¶

The article introduces the megastudy as a methodological innovation for applied behavioural science: "a massive field experiment in which the effects of many different interventions are compared in the same population on the same objectively measured outcome for the same duration." It was created "to address this limitation and accelerate the pace of discovery" arising from the lack of comparability of individual investigations that test different ideas in different samples with different outcomes and time intervals.

Design Implications¶

Context¶

Requirements¶

• Many independent scientist teams designing interventions that are deployed in the same population on the same objectively measured outcome for the same duration.

Constraints¶

• The article motivates the method by the lack of comparability of typical individual investigations, which test different intervention ideas in different samples using different outcomes over different time intervals.

Target Learners¶

• Adult members of an American fitness chain (the demonstrated population)

Target Learning Objectives¶

• Behaviour change, here encouraging sustained physical exercise habits

Claims¶

• Forecasts by impartial judges failed to predict which interventions would be most effective [+W]
• 45% of the 54 exercise interventions significantly increased weekly gym visits by 9% to 27% during the four-week intervention [+W]
• Among interventions that increased exercise during the intervention, carry-over effects were proportionally similar to previous research [~W]
• Only 8% of the exercise interventions induced behaviour change that was significant and measurable after the four-week intervention [~W]
• The top-performing exercise intervention offered microrewards for returning to the gym after a missed workout [+W]

Related Theories¶

Examples¶

-

Key Sources¶

• Kautz, T., Milkman, K. L., Gromet, D., et al. as printed in full author line. (2021). Megastudies Improve the Impact of Applied Behavioural Science. Nature, vol. 600. https://www.mathematica.org/publications/megastudies-improve-the-impact-of-applied-behavioural-science. doi:10.1038/s41586-021-04128-4