Four recommendations for using state tests in education experiments: gauge alignment, ensure reliability, collect baseline data, and consider combining results carefully¶

Strategy · All strategies
Evidence · no claims cited

Description¶

The paper's concluding chapter distills its guidance into four numbered recommendations for researchers considering state test data in RCTs. First, gauge the alignment of specific assessments with the outcome objectives of, and research questions about, the intervention of interest. Second, ensure the assessment is reliable and appropriate for the study target population. Third, whenever possible, collect and use baseline measures. Fourth, carefully consider whether and how to combine results based on distinct assessments, given variation in content and proficiency standards across grades and states.

Design Implications¶

Context¶

Requirements¶

• Researchers should review the test's technical documentation and consult the state office of assessment or test developer about difficulty for the target population

Constraints¶

• Combining results across distinct assessments requires assumptions about rescaling that study reports typically do not discuss or verify

Target Learners¶

• Education researchers designing randomized experiments

Target Learning Goals¶

• Valid and reliable estimation of intervention impacts using state assessment data

Affordances¶

• Suitability Feasibility Framework State Tests Rct

Related Strategies¶

Examples¶

-

Key Sources¶

• May, Henry, Irma Perez-Johnson, Joshua Haimson, Samina Sattar, and Phil Gleason (2009). Using State Tests in Education Experiments: A Discussion of the Issues (NCEE 2009-013). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. https://www.mathematica.org/publications/using-state-tests-in-education-experiments-a-discussion-of-the-issues