Baseline covariate adjustment in education randomized trials

Research Method · All research methods
Evidence · 5 claims (3 for, 2 mixed) · 4 studies (3 theoretical, 1 associational), q1–q2 · 0 of 4 report an effect size · 5 claims rest on one study

Description

The paper recommends that studies using student achievement outcomes generally aim to collect and use baseline data, because analyses using more than one wave of data typically have much greater statistical power than posttest-only analyses. Using prior state test results as baseline covariates can substantially reduce overall study costs, since the power improvement allows the sample size to be greatly reduced while maintaining the target power level. Baseline data also help establish treatment-control equivalence and assess potential nonresponse bias.

Accounts

• Collect and use baseline state test data as covariates to increase statistical power and reduce study costs: The paper recommends that studies using student achievement outcomes generally aim to collect and use baseline data, because analyses using more than one wave of data typically have much greater statistical power than posttest-only analyses. Using prior state test results as baseline covariates can substantially reduce overall study costs, since the power improvement allows the sample size to be greatly reduced while maintaining the target power level. Baseline data also help establish treatment-control equivalence and assess potential nonresponse bias. (May et al. (2009))
• In education RCTs, prefer impact estimators that include late pretests over those that exclude them, provided early-year impact growth is not very rapid: Design principle for RCT analysis: when pretests can only be collected after random assignment, analysts should generally use estimators that include the late pretest data rather than dropping them or substituting other baseline scores, because the precision gains typically outweigh the bias risk. The paper grounds this in a "loss function approach grounded in the causal inference literature" and conditions it on impact growth rates. (Peter Z. Schochet (2008))
• Evidence syntheses of practice-schedule effects should ensure or stratify by baseline group equivalence before pooling post-test comparisons: This principle holds that reviews comparing practice conditions must verify that groups were equivalent at pre-test, or otherwise analyze pre-post deltas rather than raw post-tests, because unequal starting performance can be mistaken for an intervention effect. The article recommends that "reviews adhere to more rigorous inclusion criteria or conduct stratified/sensitivity analyses specifically designed to account for baseline imbalances in order to provide more valid, less biased and interpretable effect estimate". The authors implemented this by excluding studies without verified baseline comparability and by analyzing changes across phases. (Ammar et al. (2025))

Claims

• Unreliability in an outcome measure reduces the statistical power of treatment-control comparisons [+M]
• In RCTs of interventions to improve student test scores, estimators including late pretests will typically be preferred to estimators excluding them or using other baseline data [+M]
• The late-pretest estimator preference holds as long as test score impacts do not grow very quickly early in the school year [~M]
• Pretest-posttest correlation coefficients for state assessment data vary by student achievement level (low-performing, average-performing, proficient). [~W]
• Randomization does not ensure baseline equivalence in small-sample motor learning studies, so unbalanced pre-tests can overstate random-practice benefits [+W]

Related Research Methods

-

Key Sources

• May, Henry, Irma Perez-Johnson, Joshua Haimson, Samina Sattar, and Phil Gleason (2009). Using State Tests in Education Experiments: A Discussion of the Issues (NCEE 2009-013). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. http://ncee.ed.gov
• Peter Z. Schochet. (2008). The Late Pretest Problem in Randomized Control Trials of Education Interventions. Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. https://ies.ed.gov/ncee
• Ammar, A., Trabelsi, K., Salem, A., Jahrami, H., & Schöllhorn, W. I. (2025). Advancing Contextual Interference: Addressing Methodological Debates, Reflecting on Meta-Analytic Practices and Generalizability, and Guiding Future Directions in Motor Learning. Educational Psychology Review. https://doi.org/10.1007/s10648-025-10043-1

Page metadata

Field	Value
type	research-method
id	baseline-covariate-adjustment-in-education-randomized-trials
title	Baseline covariate adjustment in education randomized trials
description	The paper recommends that studies using student achievement outcomes generally aim to collect and use baseline data, because analyses using more than one wave of data typically have much greater statistical power than posttest-only analyses.
status	review
generated	by process:settle-candidates on 2026-10-09
trust	unverified — no one has confirmed this page's content

Frontmatter is the page's machine-readable record — see the Schema & Guide.