Multi-armed RCT design-based framework for examining multiple interventions in a single study¶

Theory · All theories
Evidence · 1 claim (1 for) · 1 study (1 theoretical), q2 · 0 of 1 report an effect size · 1 claim rests on one study

Description¶

The report presents a design-based framework for multi-armed randomized controlled trials, which are "becoming increasingly common in social policy research to simultaneously examine the effects of multiple interventions in a single study." The framework extends two-group design-based estimation to settings where several research groups are compared in one experiment, addressing estimation, hypothesis testing, and causal identification for compliers.

Design Implications¶

Context¶

Requirements¶

• Research groups must be defined as pairwise contrasts within a single randomized study

Constraints¶

• The framework applies to design-based estimation in social policy research settings with multiple intervention arms

Target Learners¶

• Education researchers and evaluation methodologists

Target Learning Objectives¶

• Designing and analyzing rigorous evaluations of multiple educational interventions

Claims¶

• Variance terms in multi-armed RCT estimators need slight adjustment under the finite-population framework, which can reduce precision [+W]

Related Theories¶

• Design-based estimation framework extended from two-group RCTs to multi-armed RCTs
• A framework that balances Type I and Type II errors for addressing the multiple testing problem in social policy impact evaluations

Examples¶

• Empirical example using data from a multi-armed education RCT

Key Sources¶

• Peter Z. Schochet. (2017). Multi-Armed RCTs: A Design-Based Framework. Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. https://www.mathematica.org/publications/multi-armed-rcts-a-design-based-framework