National experimental study of reading and mathematics software products (NCLB-mandated first student cohort)¶

Element · All elements
Evidence · 4 claims (2 for, 2 mixed) · 2 studies (2 causal), q2–q3 · 0 of 2 report an effect size · 4 claims rest on one study

Description¶

A national study called for by the No Child Left Behind Act to assess the effectiveness of educational technology. It "identified reading and mathematics software products based on prior evidence of effectiveness and other criteria and used an experimental design", randomly assigning volunteering teachers to use or not use selected products and measuring effects on test scores after one year.

Design Implications¶

Context¶

Requirements¶

• Volunteering teachers randomly assigned to use or not use selected products
• Products selected based on prior evidence of effectiveness and other criteria

Constraints¶

• Findings reported here cover the first student cohort and one year of use

Target Learners¶

• first-grade and fourth-grade students in U.S. classrooms using reading or mathematics software

Target Learning Goals¶

• reading achievement
• mathematics achievement

Claims¶

• After one year, reading and mathematics software products did not increase or decrease test scores by amounts statistically different from zero [+W]
• Effects of mathematics software products were uncorrelated with classroom and school characteristics [~W]
• In the 2005-2006 school year, only one of 10 computer-based reading and math software products had statistically significant positive effects on student outcomes [+W]
• Effects of reading software products were correlated with student-teacher ratio in first-grade classrooms and with amount of product use in fourth-grade classrooms [~W]

Related Elements¶

• Mathematica's congressionally mandated study of educational technology (reading and math software products, two student cohorts)

Examples¶

-

Key Sources¶

• Dynarski, M., Agodini, R., Heaviside, S., Novak, T., Carey, N., Campuzano, L., Means, B., Murphy, R., Penuel, W., Javitz, H., Emery, D., & Sussex, W. (2007). Effectiveness of Reading and Mathematics Software Products: Findings from the First Student Cohort. Report to Congress. https://www.mathematica.org/publications/effectiveness-of-reading-and-mathematics-software-products-findings-from-the-first-student-cohort