Bayesian hypothesis testing as an approach to the multiple testing problem¶

Theory · All theories
Evidence · no claims cited

Description¶

The report treats Bayesian hypothesis testing as one of the approaches addressed for handling multiple testing in education impact evaluations. The abstract lists it among the report's contents: "the report provides details on the nature of the multiple testing problem and the statistical solutions that have been proposed; the creation of composite outcomes measures; and the Bayesian hypothesis testing approach". It is presented as an alternative to frequentist correction procedures.

Design Implications¶

Context¶

Requirements¶

• Applied within impact evaluations facing multiple hypothesis tests

Constraints¶

Target Learners¶

• Education researchers and evaluators

Target Learning Objectives¶

• Choosing and applying testing approaches that manage type I error across multiple comparisons

Claims¶

Related Theories¶

• A Bayesian approach to factorial design using hierarchical priors and partial pooling
• Bayesian adaptive approach to randomized policy evaluations for estimating heterogeneous treatment effects
• Bayesian adaptive randomized program evaluation framework
• A framework that balances Type I and Type II errors for addressing the multiple testing problem in social policy impact evaluations

Examples¶

• Use the BASIE framework to interpret impact estimates from evaluations

Key Sources¶

• Peter Z. Schochet. (2008). Guidelines for Multiple Testing in Impact Evaluations of Educational Interventions. Princeton, NJ: Mathematica Policy Research. https://www.mathematica.org/publications/guidelines-for-multiple-testing-in-impact-evaluations-of-educational-interventions