Bayesian approach to factorial design using hierarchical priors and partial pooling¶

Theory · All theories
Evidence · no claims cited

Description¶

The paper introduces a Bayesian framework for factorial experiments in education research. Rather than relying only on basic treatment-control comparisons, the approach uses "hierarchical priors and partial pooling" to share information across factor levels, which the authors show increases the precision of estimates in complex experiments with many factors and factor levels. It is offered as a methodological alternative to traditional statistical inference for studying multiple related interventions.

Design Implications¶

Context¶

Requirements¶

• A factorial experiment with multiple factors and factor levels whose effects can be partially pooled under hierarchical priors
• A factorial experiment with multiple factors and factor levels whose effects and interactions are of interest

Constraints¶

• The precision advantage is demonstrated for complex experiments with many factors and factor levels

Target Learners¶

• Education researchers designing experiments

Target Learning Objectives¶

• Designing feasible experiments to test multiple related interventions or implementation approaches
• designing feasible factorial experiments to test related interventions and implementation approaches

Claims¶

Related Theories¶

• Bayesian hypothesis testing as an approach to the multiple testing problem

Examples¶

• Simulation-based power calculation procedure comparing classical and Bayesian MDEs across factorial designs
• Use Bayesian inference, equivalence testing, and multiple operationalizations to make null effects theoretically informative

Key Sources¶

• Daniel Kassler, Ira Nichols-Barrer, Mariel Finucane. (2018). Beyond "Treatment versus Control": How Bayesian Analysis Makes Factorial Experiments Feasible in Education Research. Working Paper 61. Cambridge, MA: Mathematica Policy Research. https://www.mathematica.org/publications/beyond-treatment-versus-control-how-bayesian-analysis-makes-factorial-experiments-feasible
• Daniel Kassler, Ira Nichols-Barrer, Mariel Finucane. (2019). Beyond “Treatment Versus Control”: How Bayesian Analysis Makes Factorial Experiments Feasible in Education Research. Evaluation Review (online ahead of print). https://www.mathematica.org/publications/ja-beyond-treatment-versus-control-how-bayesian-analysis-makes-factorial-experiments-feasible