Butterfly Phonics improved reading comprehension of struggling Year 7 readers more than business-as-usual, with an effect size of 0.43 equivalent to five months' progress, but the effect was below the trial's minimum detectable effect size¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r2 · q3 · i2 medium
Currency · newest study 2015 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i2 Pupils receiving Butterfly Phonics had significantly higher NGRT reading comprehension scores than control pupils after adjusting for baseline, with an effect size of 0.43 (95% CI 0.03, 0.84), equivalent to five months of reading progress. → Christine Merrell 2015 q3 i? Because the observed effect size was lower than the trial's minimum detectable effect size of 0.63, the evaluators could not confidently conclude the effect was real rather than due to chance. → Christine Merrell 2015 (2)

Evidence¶

Christine Merrell 2015¶

Christine Merrell, Adetayo Kasim. (2015). Butterfly Phonics: Evaluation Report and Executive Summary. Education Endowment Foundation. https://eric.ed.gov/?id=ED581118

q3 · i2 · causal · r2

Intention-to-treat analysis of a randomised controlled trial (individual pupil randomisation within six London schools) using multilevel modelling; intervention N=161, control N=149. The effect size of 0.43 (0.03, 0.84) was "equivalent to five months of reading progress".

"There was a significant effect for the intervention on the NGRT scores, with an effect size of 0.43 (0.03, 0.84) (see Table 9). Pupils in the intervention group had, on average, higher NGRT scores than those in the control group after adjusting for baseline scores."

Christine Merrell 2015 (2)¶

Christine Merrell, Adetayo Kasim. (2015). Butterfly Phonics: Evaluation Report and Executive Summary. Education Endowment Foundation. https://eric.ed.gov/?id=ED581118

q3 · i? · causal · r2

Power analysis reported in the Methodology section: the original MDES of 0.22 ignored intra-cluster correlation; accounting for pupils nested within schools raised the MDES to 0.63, larger than the observed effect of 0.43.

"The reported MDES of 0.22 was calculated under the assumption that the data was independent. However, accounting for the nested nature of pupils within schools resulted in a bigger MDES of 0.63, partly due to heterogeneity between schools."

Discussion¶

Related Claims¶

• The cost of Butterfly Phonics as delivered in the trial was estimated at £108.50 per pupil, assuming eight pupils per class — related
• For pupils entitled to free school meals, Butterfly Phonics showed no statistically significant effects on any outcome (NGRT g = 0.16, SWRT g = 0.42, PhAB g = 0.18) — related
• Butterfly Phonics showed positive but non-statistically-significant effects on secondary outcome measures of single word reading (g = 0.38) and non-word reading (g = 0.23) — related