Across 82 EEF RCT evaluations, overall meta-analysis weighted mean effect sizes were small: +0.04 SD for primary ITT, +0.01 SD for secondary ITT, and +0.03 SD for FSM subsample attainment outcomes¶

Claim · All claims
Evidence · 1 study (3 entries) · quant-synthesis r? · q3 · i0 negligible
Currency · newest study 2021 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i0 The 82 evaluations reported 133 primary ITT effect sizes with an overall meta-analysis weighted mean of +0.04 SD. → Demack 2021 q3 i0 The 35 evaluations reporting secondary ITT outcomes yielded 78 effect sizes with an overall weighted mean of +0.01 SD. → Demack 2021 (2) q3 i0 The 73 evaluations reporting FSM subsample outcomes yielded 149 effect sizes with an overall weighted mean of +0.03 SD. → Demack 2021 (3)

Evidence¶

Demack 2021¶

Demack, S., Maxwell, B., Coldwell, M., Stevens, A., Wolstenholme, C., Reaney-Wood, S., Stiell, B., & Lortie-Forgues, H. (2021). Review of EEF projects: Evaluation report. Education Endowment Foundation. https://eric.ed.gov/?id=ED625480

q3 · i0 · quant-synthesis · r?

Meta-analysis of effect sizes from the first 82 EEF RCT evaluations published up to January 2019. The review reports "an overall meta-analysis weighted mean effect size of +0.04 standard deviations (SD)" for primary ITT attainment outcomes.

"A total of 133 primary ITT effect sizes were reported by the 82 evaluations in the review and an overall meta-analysis weighted mean effect size of +0.04 standard deviations (SD) was observed."

Demack 2021 (2)¶

Demack, S., Maxwell, B., Coldwell, M., Stevens, A., Wolstenholme, C., Reaney-Wood, S., Stiell, B., & Lortie-Forgues, H. (2021). Review of EEF projects: Evaluation report. Education Endowment Foundation. https://eric.ed.gov/?id=ED625480

q3 · i0 · quant-synthesis · r?

Meta-analysis of secondary ITT attainment effect sizes from 35 of the 82 EEF evaluations. The weighted mean was "+0.01 SD", smaller than the primary outcome mean.

"A total of 78 secondary ITT attainment effect sizes were reported by 35 of the 82 evaluations in the review and an overall meta-analysis weighted mean effect size of +0.01 SD was observed."

Demack 2021 (3)¶

Demack, S., Maxwell, B., Coldwell, M., Stevens, A., Wolstenholme, C., Reaney-Wood, S., Stiell, B., & Lortie-Forgues, H. (2021). Review of EEF projects: Evaluation report. Education Endowment Foundation. https://eric.ed.gov/?id=ED625480

q3 · i0 · quant-synthesis · r?

Meta-analysis of FSM (free school meals) subsample attainment effect sizes from 73 of the 82 EEF evaluations. The weighted mean was "+0.03 SD".

"A total of 149 FSM subsample attainment effect sizes were reported by 73 of the 82 evaluations in the review and an overall meta-analysis weighted mean effect size of +0.03 SD was observed."

Discussion¶

Related Claims¶

• Cost effectiveness across EEF trials was highly skewed, with a mean of £150 and median of £54 per pupil for an effect size of +0.10 SD among the 40 evaluations showing positive impact — related
• Interventions with an English curriculum focus were associated with higher primary ITT effect sizes than maths or cross-curriculum focus, while maths focus was associated with higher secondary ITT effect sizes — a narrower finding that bears on this claim
• Interventions located in one or up to three geographical areas showed higher primary ITT and FSM effect sizes than interventions spread across more areas — related
• Psychological outcomes reported by 21 EEF evaluations showed an overall meta-analysis weighted mean effect size of +0.05 SD, but were excluded from the main meta-analyses — related
• Effect sizes were higher for interventions with a total cost between £250k and £750k than for interventions that were more or less expensive — related
• EEF-funded interventions showed no evidence of positive or negative overall effects on FSM pupils' mathematics outcomes — a narrower finding that bears on this claim
• EEF-funded interventions improved FSM pupils' literacy outcomes with a pooled effect size of 0.06, equivalent to about one month's progress — a narrower finding that bears on this claim
• EEF interventions showed a small overall narrowing of the literacy attainment gap in favour of FSM pupils, dependent on Key Stage — related
• TA-led interventions were associated with higher primary ITT and FSM effect sizes than other interventions, a pattern not observed for secondary ITT effect sizes — related
• Mathematics effects on FSM pupils differed by outcome status: near-zero as primary outcome (0.01) and negative as secondary outcome (-0.07) — related
• Small group and one-to-one interventions produced larger literacy gains for FSM pupils than whole class or whole school approaches — related