When a domain-identical pretest is infeasible, use cross-domain pretests, fluid intelligence, or sociodemographic covariates as meaningful alternatives¶

Strategy · All strategies
Evidence · 3 claims (2 for, 1 mixed) · 2 studies (1 quant-synthesis, 1 review), q3 · 1 of 2 report an effect size · 3 claims rest on one study

Description¶

The article recommends that researchers planning randomized trials first seek the latest domain-identical pretest, but recognizes practical situations where IP cannot be collected, such as multiple outcome domains with limited testing time, outcomes that change after the trial starts, or pretests that presuppose intensive instruction. In such situations, "CP, Gf, or SC may be meaningful alternatives to IP", and the compiled design parameters quantify the precision costs of each substitute.

Design Implications¶

Context¶

Requirements¶

• A priori, ideally preregistered covariate decisions justified by substantive theory and empirical results tied to the specific research field

Constraints¶

• IP remains the strongest covariate type in general; alternatives carry lower single-level prognostic power and their multilevel evidence base for Gf was lacking before this study

Target Learners¶

• Students in grades 1-12 participating in educational randomized trials

Target Learning Goals¶

• Maximizing design sensitivity when planning individually or cluster-randomized trials on student achievement

Claims¶

• Domain-identical pretests are the most powerful single-level covariate type for student achievement, outperforming cross-domain pretests, fluid intelligence, and sociodemographic characteristics [+W]
• Sociodemographic covariates outweigh domain-identical pretests for predicting between-classroom and between-school variance in elementary school [~W]
• General and fluid intelligence correlate strongly with mathematics achievement (r = .41 to .62), and the fluid-intelligence correlation is larger in older samples [+M]

Related Strategies¶

Examples¶

-

Key Sources¶

• Stallasch, S. E., Lüdtke, O., Artelt, C., Hedges, L. V., & Brunner, M. (2024). Single- and Multilevel Perspectives on Covariate Selection in Randomized Intervention Studies on Student Achievement. Educational Psychology Review. https://doi.org/10.1007/s10648-024-09898-7