Link implementation and impact findings when analyzing and reporting impact studies¶

Strategy · All strategies
Evidence · no claims cited

Description¶

The guide recommends that analyses and reporting connect implementation data to impact estimates: it 'presents strategies for analysis and reporting about these topics, and also for linking implementation and impact findings'. This enables educators to judge whether a tested intervention is likely to fit their own settings based on how implementation related to outcomes.

Design Implications¶

Context¶

Requirements¶

• Both implementation data and impact estimates must be available in the study for the linkage strategies to apply.

Constraints¶

Target Learners¶

• Education researchers conducting impact studies

Target Learning Goals¶

• Analyze and report relationships between implementation and intervention impacts

Related Strategies¶

• Assess whether and how the planned intervention is implemented
• Document the implementation context and measure the intervention-control contrast
• Specify implementation research questions as a distinct step in impact study design
• Follow a step-by-step estimation procedure with example-based computer code when analyzing PN-RCT data
• Report evaluation results with implementation and context details so schools can judge fit, and use measures educators value

Examples¶

-

Key Sources¶

• Hill, C. J., Scher, L., Haimson, J., & Granito, K. (2023). Conducting Implementation Research in Impact Studies of Education Interventions: A Guide for Researchers. Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. https://ies.ed.gov/ncee/