Three-dataset within-study comparison design varying pretest-selection correlation¶

Element · All elements
Evidence · 1 claim (1 for) · 1 study (1 causal), q2 · 0 of 1 report an effect size · 1 claim rests on one study

Description¶

The article's evidence base is a design combining "two within study comparisons and an especially strong quasi-experiment," each featuring "an educational intervention that seeks to improve achievement." Across the three datasets with two outcomes each, the design yields six instances in which the correlation between the pretest and the selection process is varied, with three low-correlation and three high-correlation cases, allowing within-study examination of how pretest measures reduce selection bias.

Design Implications¶

Context¶

Requirements¶

• Requires datasets where both pretest and post-intervention measures of the outcome and information on the selection process are available

Constraints¶

Target Learners¶

• Methodologists and education researchers evaluating observational studies

Target Learning Goals¶

• Assessing and eliminating selection bias in estimates of educational intervention effects

Claims¶

• In the qualifying quasi-experimental study, at-risk Kindergarten students using Waterford Level One for 15-minute daily sessions were matched to comparison students and pretested in fall and posttested in spring [+W]

Related Elements¶

Examples¶

-

Key Sources¶

• Kelly Hallberg, Thomas D. Cook, Peter M. Steiner, M. H. Clark. (2018). Pretest Measures of the Study Outcome and the Elimination of Selection Bias: Evidence from Three Within Study Comparisons. Prevention Science, vol. 19, issue 3. https://www.mathematica.org/publications/pretest-measures-of-the-study-outcome-and-the-elimination-of-selection-bias-evidence-from-three