Use value-added school ratings for low-cost ex post facto evaluation of district programs¶

Strategy · All strategies
Evidence · no claims cited

Description¶

The article recommends evaluating already-implemented programs by comparing schools' poverty-adjusted effectiveness ratings before and after implementation, using data districts already collect. It states that "Under the right circumstances, models such as the one used here allow an ex post facto evaluation of program effectiveness," citing advantages of cost (existing data and common spreadsheets), no required behavior change, and flexibility to accommodate new tests or changed reporting systems. It also recommends that reliable data on school practices be collected alongside achievement data, suggesting qualitative classroom observation could help connect practice to achievement.

Design Implications¶

Context¶

Requirements¶

• Systematic recording of which schools implement which programs, which the article says currently does not exist

Constraints¶

• Requires enough implementing schools for analysis but not so many that no control group remains; biased toward programs targeting heavily tested grades such as fourth grade; changes in tested years or subjects can create apparent change where none exists; outside changes may bias results

Target Learners¶

• district administrators evaluating school-level programs

Target Learning Goals¶

• raising student achievement on standardized tests, particularly among low-income and minority students

Affordances¶

• Poverty Adjusted School Effectiveness Rating Model

Related Strategies¶

• Use LoU data to monitor implementation, plan staff development, select research samples, and evaluate programs

Examples¶

-

Key Sources¶

• Bruce Thompson. (2006). Evaluating Three Programs Using a School Effectiveness Model: Direct Instruction, Target Teach, and Class Size Reduction. Third Education Group Review, Vol. 2, No. 3. https://eric.ed.gov/?id=EJ874476