The DreamBox effect on the MAP Computation subtest was not statistically significant after WWC multiple-comparison correction¶

Claim · All claims
Evidence · 1 study · causal r3 · q3 · i0 negligible
Currency · newest study 2013 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i0 The MAP Computation subtest showed a raw effect size of 0.15 favoring DreamBox Learning (p = 0.03), but a multiple-comparison correction set a critical p-value of 0.01, so the WWC does not find this result statistically significant. → computation

Evidence¶

computation¶

U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2013, December). Elementary School Mathematics intervention report: DreamBox Learning. Retrieved from https://ies.ed.gov/ncee/wwc/InterventionReport/627

q3 · i0 · causal · r3

Supplemental subscale findings from the same RCT (n = 546 for this subtest): intervention mean 163.00 (SD 20.70) versus comparison 158.80 (SD 19.50), effect size 0.15, improvement index +6, p = 0.03. The report notes "the WWC does not find these results to be statistically significant" after correction.

"MAP: Computation subtest Grades K and 1 546 students 163.00 (20.70) 158.80 (19.50) 4.20 0.15 +6 0.03"

Discussion¶

Related Claims¶

• The DreamBox effect on the MAP Measurement and Geometry subtest was not statistically significant after WWC multiple-comparison correction — related
• DreamBox Learning produced a statistically significant positive effect on MAP overall math scores for K–1 students — related
• DreamBox effects on the Problem Solving, Number Sense, and Statistics and Probability subtests were not statistically significant — related
• Author-reported statistically significant program effects did not remain significant after WWC corrections for clustering and multiple comparisons — a broader claim this one bears on
• In a randomized study in three high-poverty Bay Area charter schools, K-1 students using DreamBox gained significantly more than control students on the NWEA MAP Measurement and Geometry subtest (ES +0.16) and scored higher overall (ES +0.11) — related