Bayesian adaptive randomized program evaluation framework¶

Theory · All theories
Evidence · 1 claim (1 for) · 1 study (1 theoretical), q2 · 0 of 1 report an effect size · 1 claim rests on one study

Description¶

The article presents a Bayesian approach to randomized program evaluations in which the design "adapts to accumulating evidence": over the course of an evaluation, "more study subjects are allocated to treatment arms that are more promising, given the specific subgroup from which each subject comes". The framework is offered as a novel statistical method for evaluators seeking to take full advantage of growing public-program data streams, efficiently identifying what works for whom.

Design Implications¶

Context¶

Requirements¶

• Randomized program evaluation with multiple treatment arms and accumulating data streams

Constraints¶

• Superior inference over standard designs holds under conditions the authors identify, not universally

Target Learners¶

• public program evaluators
• education and health program researchers

Target Learning Objectives¶

• identifying what works for whom in public program evaluation
• efficient causal inference from program data

Claims¶

• Bayesian Adaptive Benefits Study Subjects [+M]

Related Theories¶

• Bayesian adaptive approach to randomized policy evaluations for estimating heterogeneous treatment effects
• Bayesian hypothesis testing as an approach to the multiple testing problem

Examples¶

• Use Bayesian adaptive designs to channel growing public-program data streams into evaluation
• Simulation study of Bayesian adaptive design conditions
• Match each study subject to the candidate intervention most likely to benefit them

Key Sources¶

• Mariel McKenzie Finucane, Ignacio Martinez, Scott Cody. (2018). What Works for Whom? A Bayesian Approach to Channeling Big Data Streams for Public Program Evaluation. American Journal of Evaluation, vol. 39, issue 1. https://journals.sagepub.com/loi/aje