The Boston replication reproduced significant effects on the TEMA-3, ASPENS, and RAENS and newly found significant effects on the NSB and SESAT, with effect sizes within or exceeding the Oregon study's bounds¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r2–r3 · q3 · i1–i2
Currency · newest study 2016 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i2 Statistically significant effects on the TEMA-3, ASPENS, and RAENS replicated in Boston, with effect sizes ranging from 0.31 to 1.08. → Doabler 2016 q3 i1 The NSB and SESAT, non-significant in Oregon, showed statistically significant impacts in Boston (g = .40 and .24). → Doabler 2016 (2)

Evidence¶

Doabler 2016¶

Doabler, C. T., Clarke, B., Kosty, D. B., Kurtz-Nelson, E., Fien, H., Smolkowski, K., & Baker, S. K. (2016). Testing the efficacy of a tier 2 mathematics intervention: A conceptual replication study. Exceptional Children, 83, 92–110. https://doi.org/10.1177/0014402916660084

q3 · i2 · causal · r3

Cross-study comparison reported in the Discussion: the replication, conducted in a different region with a stronger counterfactual, reproduced the Oregon trial's significant effects on three measures, with effect sizes "ranged from 0.31 to 1.08"; all but the RAENS fell within the Oregon confidence intervals, and the RAENS exceeded the upper bound.

"Overall, statistically significant effects on the TEMA-3, ASPENS, and RAENS were replicated in the Boston study. The effect sizes obtained for these measures ranged from 0.31 to 1.08."

Doabler 2016 (2)¶

Doabler, C. T., Clarke, B., Kosty, D. B., Kurtz-Nelson, E., Fien, H., Smolkowski, K., & Baker, S. K. (2016). Testing the efficacy of a tier 2 mathematics intervention: A conceptual replication study. Exceptional Children, 83, 92–110. https://doi.org/10.1177/0014402916660084

q3 · i1 · causal · r2

Discussion comparison of the two trials: the Oregon study found no significant NSB or SESAT differences, whereas the Boston replication did, with effect sizes of .40 and .24 respectively. The review of these contrasts is the authors' own account of their two trials.

"In the Oregon study, Clarke and colleagues (2016) failed to find statistically significant differences on the NSB or the SESAT. Notably, however, results from the Boston study indicated a statistically significant impact on these two distal outcome measures."

Discussion¶

Related Claims¶

• Kindergartners with mathematics difficulties randomly assigned to the ROOTS intervention made significantly greater fall-to-spring gains than control students on the NSB, ASPENS, TEMA-3, and RAENS — related
• ROOTS students also had higher covariate-adjusted SESAT posttest scores than control students — related
• Rates of treatment responsiveness on the TEMA-3 were similar across the Oregon and Boston ROOTS studies — related
• No statistically significant differences between ROOTS and control conditions on oral counting gains or covariate-adjusted SAT-10 scores — related
• The ROOTS intervention produced greater fall-to-spring kindergarten gains than control instruction on four of five outcome measures for at-risk students — related
• ROOTS effects on mathematics achievement were not detected on first-grade follow-up measures — related
• An earlier preliminary investigation of ROOTS found small-to-medium positive effects on kindergarten mathematics achievement measures — related
• ROOTS students made greater gains than their not-at-risk peers, reducing the achievement gap — related
• Year 1 of the ROOTS efficacy trial used a classroom-blocked RCT assigning at-risk kindergartners to ROOTS-large, ROOTS-small, or no-treatment control conditions — related
• At-risk kindergarten students receiving the ROOTS Tier 2 intervention made significantly greater mathematics gains than at-risk control peers on the TEMA standard score — related
• The Pre-K Mathematics intervention produces greater math knowledge gains in 4-year-olds than business-as-usual control — related
• The ROOTS kindergarten mathematics intervention significantly improved fall-to-spring gains on three of six outcome measures compared with control, with positive effect sizes on all measures — related