Language and early writing interventions showed larger effects on researcher-developed than standardized outcome measures, while phonological awareness, print knowledge, and decoding effects were comparable across measure types¶

Claim · All claims
Evidence · 1 study (3 entries) · quant-synthesis r2 · q3 · i1–i2
Currency · newest study 2021 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i2 The weighted effect size for language was significantly larger on researcher-developed measures than standardized ones (0.43 versus 0.12). → Sarah Herrera 2021 q3 i? Phonological awareness effects were statistically comparable across measure types (0.43 researcher-developed versus 0.28 standardized). → Sarah Herrera 2021 (2) q3 i? Early writing improved significantly on researcher-developed measures (0.36) but the standardized effect (0.23) was not statistically significant. → Sarah Herrera 2021 (3)

Evidence¶

Sarah Herrera 2021¶

Sarah Herrera, Beth M. Phillips, Yi-Chieh Newton, Jennifer L. Dombek, and James A. Hernandez. (2021). Effectiveness of Early Literacy Instruction: Summary of 20 Years of Research. Regional Educational Laboratory Southeast, REL 2021-084. https://ies.ed.gov/ncee/edlabs

q3 · i2 · quant-synthesis · r2

Figure 4 comparison across 74 language-teaching interventions (31 researcher-developed, 53 standardized measures). Weighted effect sizes were 0.43 for researcher-developed versus 0.12 for standardized measures, a statistically significant difference the review attributes to researcher-developed measures representing skills more similar to those taught.

"The weighted effect size was significantly larger for researcher-developed outcome measures than for standardized ones (0.43 versus 0.12; figure 4; see also table B7 in appendix B)"

Sarah Herrera 2021 (2)¶

Sarah Herrera, Beth M. Phillips, Yi-Chieh Newton, Jennifer L. Dombek, and James A. Hernandez. (2021). Effectiveness of Early Literacy Instruction: Summary of 20 Years of Research. Regional Educational Laboratory Southeast, REL 2021-084. https://ies.ed.gov/ncee/edlabs

q3 · i? · quant-synthesis · r2

Comparison across 36 phonological awareness interventions in table B10. The review reports the difference "was not statistically significant," suggesting phonological awareness interventions improve performance regardless of outcome measure type.

"the weighted effect size was 0.43 for researcher-developed outcome measures and 0.28 for standardized outcome measures; the difference was not statistically significant"

Sarah Herrera 2021 (3)¶

Sarah Herrera, Beth M. Phillips, Yi-Chieh Newton, Jennifer L. Dombek, and James A. Hernandez. (2021). Effectiveness of Early Literacy Instruction: Summary of 20 Years of Research. Regional Educational Laboratory Southeast, REL 2021-084. https://ies.ed.gov/ncee/edlabs

q3 · i1 · quant-synthesis · r2

Comparison across 11 early writing interventions (7 researcher-developed, 4 standardized measures) in figure 9. The researcher-developed weighted effect size was 0.36, equivalent to a 14 percentile point increase; the standardized effect of 0.23 was not statistically significant.

"interventions that taught early writing significantly improved early writing performance on researcher-developed outcome measures, among the 11 interventions evaluated in high-quality impact studies and included at least one effect size estimate (weighted effect size of 0.36; figure 9; see also table B18"

Discussion¶

Related Claims¶

• ITS evaluation results depend strongly on test type: effects are large on locally developed tests (ES = 0.73) and small on standardized tests (ES = 0.13) — the general claim this finding bears on, with the studies behind it
• Of 132 early literacy interventions evaluated by high-quality impact studies, 38 demonstrated effectiveness in at least one language or literacy outcome domain and 93 showed inconclusive effects — a broader claim this one bears on
• Teaching phonological awareness and print knowledge improved decoding performance even without decoding instruction, and adding decoding instruction more than doubled the weighted effect size — related
• Teaching phonological awareness and print knowledge improved early writing performance with or without early writing instruction — related
• Instruction in language, phonological awareness, and decoding improved performance in the taught domain, while interventions teaching print knowledge or early writing exclusively did not — related
• Among language-only interventions, instructional practices and one-on-one or small-group delivery produced significantly larger language gains than curricula and large-group or whole-class delivery — related
• The type of CT measure employed (standardized vs. non-standardized; item format) influences evaluation of intervention effectiveness — related