Untestable-assumptions account for bias in nonexperimental estimation¶

Theory · All theories
Evidence · no claims cited

Description¶

The paper frames nonexperimental methods such as regression modeling and statistical matching as unbiased only under assumptions that "are usually not testable." This framing explains why bias in nonexperimental estimators must be assessed indirectly, for example by comparing estimates against experimental benchmarks, and motivates the paper's reassessment of when such methods perform adequately.

Design Implications¶

Context¶

Requirements¶

• Underlying assumptions of the nonexperimental method must hold for estimates to be unbiased

Constraints¶

• The assumptions are usually not testable directly, per the paper's abstract

Target Learners¶

• education researchers and evaluators

Target Learning Objectives¶

• understanding when nonexperimental evaluation methods yield unbiased treatment-effect estimates

Claims¶

Related Theories¶

• The Rothstein falsification test as a diagnostic for value-added model bias

Examples¶

-

Key Sources¶

• Kenneth Fortson, Philip Gleason, Emma Kopa, Natalya Verbitsky-Savitz. (2013). Horseshoes, Hand Grenades, and Treatment Effects? Reassessing Bias in Nonexperimental Estimators. Working Paper 16, Oakland, CA: Mathematica Policy Research. https://www.mathematica.org/publications/horseshoes-hand-grenades-and-treatment-effects-reassessing-bias-in-nonexperimental-estimators