Neither intervention showed a statistically significant effect for free-school-meal pupils¶

Claim · All claims
Evidence · 1 study · causal r3 · q3 · i0 negligible
Currency · newest study 2014 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i0 For FSM pupils, effect sizes were -0.30 for Chatterbooks and -0.09 for Chatterbooks Plus, neither statistically significant, so results should be interpreted as chance findings. → Styles 2014

Evidence¶

Styles 2014¶

Styles, B., Clarkson, R., & Fowler, K. (2014). Chatterbooks: Evaluation Report and Executive Summary. Education Endowment Foundation. https://eric.ed.gov/?id=ED581155

q3 · i0 · causal · r3

Pre-specified subgroup analysis on the primary reading outcome using free school meals eligibility as a proxy for pupil premium. Both subgroup effect sizes had confidence intervals passing through zero and were marked "No" for statistical significance; the report notes subgroup analyses are always less secure than overall findings.

"Chatterbooks (FSM pupils) 37 -0.30 (-0.63, 0.03) -4 No Chatterbooks Plus (FSM pupils) 40 -0.09 (-0.39, 0.21) -1 No"

Discussion¶

Learner Variables¶

• Socioeconomic Status — moderator: an instructional effect differs by group

Related Claims¶

• The intervention showed no evidence of differentially benefiting pupils eligible for free school meals in any of the schools — related
• The VEFP showed no statistically significant impact on reading for pupils eligible for free school meals — related
• The TextNow programme showed a differential effect by free school meal eligibility: a small non-significant positive effect for FSM-eligible pupils and a significant negative effect for non-FSM pupils — related
• FSM-eligible pupils made around five additional months' progress under TDTS (effect size +0.38), though this finding has lower security — related
• The trial found a statistically significant negative literacy impact for FSM-eligible pupils, treated with caution — related
• RLC showed no differential impact for pupils eligible for free school meals — related
• Subgroup analyses by gender, ethnicity, free school meal status and baseline ability all showed point estimates near zero with wide confidence intervals. — related
• Literacy benefits for FSM pupils were greatest in Key Stage 1 (ES 0.09) and Key Stage 3 (ES 0.08) and smallest in Key Stages 2 and 4 — related
• Chatterbooks showed no statistically significant effect on reading ability among Year 7 pupils below Level 4, with a slightly negative point estimate — related
• Chatterbooks Plus showed no statistically significant effect on reading ability, with an effect size indistinguishable from zero — related
• Secondary outcomes (sentence completion and passage comprehension subscales) showed no significant effects for either intervention — related
• The chess intervention showed no evidence of a positive impact on Key Stage 2 scores for children eligible for free school meals — related
• No evidence of improved outcomes for pupils eligible for free school meals — related
• The FSM subgroup effect of Rhythm for Reading (effect size 0.11) was not statistically significant — related
• Neither NGRT secondary outcome (sentence completion or passage comprehension) showed a statistically significant effect of Rhythm for Reading — related
• The intervention appeared more effective for FSM pupils, but the FSM interaction was not statistically significant — related
• Baseline factors of gender, free school meal status and SEN showed no significant moderation of the intervention effect — a broader claim this one bears on