Forecasts by impartial judges failed to predict which interventions would be most effective¶

Claim · All claims
Evidence · 1 study · causal r? · q3
Currency · newest study 2021 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? Impartial judges' forecasts did not successfully predict which of the 54 interventions would be most effective in the megastudy. → Kautz 2021

Evidence¶

Kautz 2021¶

Kautz, T., Milkman, K. L., Gromet, D., et al. as printed in full author line. (2021). Megastudies Improve the Impact of Applied Behavioural Science. Nature, vol. 600. https://www.mathematica.org/publications/megastudies-improve-the-impact-of-applied-behavioural-science doi:10.1038/s41586-021-04128-4

q3 · i? · causal · r?

Forecasting component of the fitness-chain megastudy: the article reports that "Forecasts by impartial judges failed to predict which interventions would be most effective." No accuracy statistic or effect size for the forecasting task is printed in the available text.

"Forecasts by impartial judges failed to predict which interventions would be most effective, underscoring the value of testing many ideas at once and, therefore, the potential for megastudies to improve the evidentiary value of behavioural science."

Discussion¶

Related Claims¶

• 45% of the 54 exercise interventions significantly increased weekly gym visits by 9% to 27% during the four-week intervention — related
• The top-performing exercise intervention offered microrewards for returning to the gym after a missed workout — related
• Only 8% of the exercise interventions induced behaviour change that was significant and measurable after the four-week intervention — related
• Among interventions that increased exercise during the intervention, carry-over effects were proportionally similar to previous research — related