A matched-comparison analysis found no effect of the TenMarks summer program on NWEA MAP math scores¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r2 · q2
Currency · newest study 2016 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? TenMarks participants showed no effect on NWEA MAP Math scores pre to post compared with a matched control group who did not use the program. → Christina Luke 2016

Evidence¶

Christina Luke 2016¶

Christina Luke, Christa M. Simone, Aubrey T. Francisco. (2016). Rapid Cycle Pilots: Summer Ed-Tech Programs. Digital Promise. https://digitalpromise.dspacedirect.org/items/8017846c-4099-4af3-92e4-462cd5d6262d doi:10.51388/20.500.12265/26

q2 · i? · causal · r2

Kettle Moraine matched-comparison analysis: 139 pilot participants versus a demographically similar control group, with a linear model controlling for baseline score, grade, gender, race, and free/reduced-price lunch status. No effect was detected; no effect size is printed.

"We found no effect of TenMarks on NWEA MAP Math scores pre and post compared to a control group who did not use the program. Because of limited student engagement, results should be treated with caution."

Christina Luke 2016 (2)¶

Christina Luke, Christa M. Simone, Aubrey T. Francisco. (2016). Rapid Cycle Pilots: Summer Ed-Tech Programs. Digital Promise. https://digitalpromise.dspacedirect.org/items/8017846c-4099-4af3-92e4-462cd5d6262d

q2 · i? · causal · r2

Table 3 prints pilot participants' mean NWEA MAP scores of 233.3 (SD 14.8) at baseline in May and 229.1 (SD 16.0) at follow-up in August, alongside control-group means of 237.1 and 234.8.

"Mean Scores- Pilot Participants 139 233.3 (14.8) 229.1 (16.0)"

Discussion¶

Related Claims¶

• Students used their assigned ed-tech tools far below recommended levels, and usage declined over the summer — related
• An earlier pilot implementation of Zearn Math without full implementation support found no significant differences versus comparison students — related
• Vista students using Mathspace showed statistically significant math gains over the summer, while benchmark scores were not significantly different in the other two districts — related
• Springboard Summer scholars show larger reading improvements than statistically equivalent non-participating peers, meeting ESSA Tier 2 moderate evidence standards — related
• The report attributes stagnant learning results to program implementation rather than to the ed-tech tools themselves — related
• Zearn Math schools significantly outperformed control schools on NWEA MAP in a large Texas RCT (effect size +0.11 overall) — related