The DreamBox effect on the MAP Measurement and Geometry subtest was not statistically significant after WWC multiple-comparison correction¶

Claim · All claims
Evidence · 1 study · causal r3 · q3 · i0 negligible
Currency · newest study 2013 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i0 The MAP Measurement and Geometry subtest showed a raw effect size of 0.15 favoring DreamBox Learning (p = 0.03), but the WWC-computed critical p-value of 0.02 means the result is not statistically significant. → measgeom

Evidence¶

measgeom¶

U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2013, December). Elementary School Mathematics intervention report: DreamBox Learning. Retrieved from https://ies.ed.gov/ncee/wwc/InterventionReport/627

q3 · i0 · causal · r3

Supplemental subscale findings from the same RCT (n = 550): intervention mean 155.50 (SD 18.30) versus comparison 151.80 (SD 18.10), effect size 0.15, improvement index +6, p = 0.03. The report states the WWC-computed critical p-value was 0.02 for this subtest, so it is not statistically significant.

"MAP: Measurement and Geometry subtest Grades K and 1 550 students 155.50 (18.30) 151.80 (18.10) 3.70 0.15 +6 0.03"

Discussion¶

Related Claims¶

• The DreamBox effect on the MAP Computation subtest was not statistically significant after WWC multiple-comparison correction — related
• DreamBox Learning produced a statistically significant positive effect on MAP overall math scores for K–1 students — related
• DreamBox effects on the Problem Solving, Number Sense, and Statistics and Probability subtests were not statistically significant — related
• Author-reported statistically significant program effects did not remain significant after WWC corrections for clustering and multiple comparisons — a broader claim this one bears on
• Measuring the impact of math instruction and interventions is key to improving students' overall math performance — related
• In a randomized study in three high-poverty Bay Area charter schools, K-1 students using DreamBox gained significantly more than control students on the NWEA MAP Measurement and Geometry subtest (ES +0.16) and scored higher overall (ES +0.11) — reports the opposite