Define the target population as the first step in designing a generalizable impact study¶

Strategy · All strategies
Evidence · no claims cited

Description¶

This strategy comes from a practical guide for researchers designing education impact studies. The guide instructs researchers to begin by "defining the target population" before any sampling or recruitment work, so that subsequent design decisions can be judged against the population to which findings are meant to apply. It is the first key step the guide enumerates for improving generalizability.

Design Implications¶

Context¶

Requirements¶

• The research team must specify the target population before selecting schools or interpreting findings

Constraints¶

Target Learners¶

• education researchers designing impact studies

Target Learning Goals¶

• designing impact studies whose findings generalize to a target population

Related Strategies¶

• Assess and adjust for differences between the study sample and the target population
• Actively manage school recruitment to protect the generalizability of impact studies
• Select a sample of schools, including replacement schools when needed, to support generalizable impact studies
• Triangulate internal-validity findings with interaction analyses, propensity-score generalizability indices, and adapted extensions to establish external validity
• Specify implementation research questions as a distinct step in impact study design

Examples¶

-

Key Sources¶

• Tipton, E., & Olsen, R. B. (2022). Enhancing the Generalizability of Impact Studies in Education. Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance (NCEE 2022-003). https://ies.ed.gov/ncee/