DreamBox Learning produced a statistically significant positive effect on MAP overall math scores for K–1 students¶

Claim · All claims
Evidence · 2 studies · 2 causal r3 · q3 · i0 negligible
Currency · newest study 2013 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i0 In an RCT with 557 kindergarten and first-grade students, DreamBox Learning students scored significantly higher than comparison students on the MAP overall math score, with an effect size of 0.11 and an improvement index of +4 percentile points. → rct-overall

Evidence¶

rct-overall¶

U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2013, December). Elementary School Mathematics intervention report: DreamBox Learning. Retrieved from https://ies.ed.gov/ncee/wwc/InterventionReport/627

q3 · i0 · causal · r3

A randomized controlled trial in three Rocketship Education charter schools in San Jose, California, randomly assigned 557 K–1 students within schools to DreamBox Learning or additional literacy instruction. The WWC reports "2.80 0.11 +4 0.04" (mean difference, effect size, improvement index, p-value) favoring the intervention group.

"Wang and Woodworth (2011) reported a statistically significant positive difference between students in the DreamBox Learning and comparison groups on the Measures of Academic Progress (MAP) overall math score."

rct-table¶

U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2013, December). Elementary School Mathematics intervention report: DreamBox Learning. Retrieved from https://ies.ed.gov/ncee/wwc/InterventionReport/627

q3 · i0 · causal · r3

WWC-calculated findings table for the mathematics achievement domain: intervention mean 159.00 (SD 16.60) versus comparison 156.20 (SD 15.10), "2.80 0.11 +4 0.04" for mean difference, effect size, improvement index and p-value, based on OLS estimates using the full sample including outliers.

"MAP: Overall math score Grades K and 1 557 students 159.00 (16.60) 156.20 (15.10) 2.80 0.11 +4 0.04"

Discussion¶

Related Claims¶

• The DreamBox effect on the MAP Computation subtest was not statistically significant after WWC multiple-comparison correction — related
• The DreamBox effect on the MAP Measurement and Geometry subtest was not statistically significant after WWC multiple-comparison correction — related
• DreamBox effects on the Problem Solving, Number Sense, and Statistics and Probability subtests were not statistically significant — related
• In the reviewed RCT, students averaged 21.8 hours of DreamBox usage, below the developer's recommended 90 minutes per week — related
• Results were more robust for students meeting the usage recommendation of 30 minutes per week over a minimum of 18 weeks — related
• Measuring the impact of math instruction and interventions is key to improving students' overall math performance — related
• Averaging across the two randomized DreamBox studies and measures, the effect size is +0.10, supporting a Strong rating with a badge for two randomized studies with significant positive results — a broader claim this one bears on
• In a randomized study in three high-poverty Bay Area charter schools, K-1 students using DreamBox gained significantly more than control students on the NWEA MAP Measurement and Geometry subtest (ES +0.16) and scored higher overall (ES +0.11) — related
• Two student-level randomized studies in California, Maine, and Minnesota high schools found significant positive impacts on NWEA MAP reading (average effect size +0.09) — related