Mathematica's congressionally mandated study of educational technology (reading and math software products, two student cohorts)¶

Element · All elements
Evidence · 3 claims (3 for) · 1 study (1 causal), q3 · 0 of 1 report an effect size · 3 claims rest on one study

Description¶

A congressionally mandated evaluation of educational technology conducted by Mathematica, examining computer-based reading and math software products across two student cohorts. The final report "presents effects for the 2005-2006 school year for 10 computer-based reading and math products," comparing product effects across the first and second year of implementation and examining whether teacher experience was related to changes in effects.

Design Implications¶

Context¶

Requirements¶

• Effects estimated for computer-based reading and math products in school-year implementations across two student cohorts

Constraints¶

• The available page reports effects for the 2005-2006 school year; only one product showed statistically significant positive effects

Target Learners¶

• K-12 students using reading and math software products

Target Learning Goals¶

• Reading achievement
• Mathematics achievement

Claims¶

• In the 2005-2006 school year, only one of 10 computer-based reading and math software products had statistically significant positive effects on student outcomes [+W]
• Software product effects changed between the first and second year of the study, warranting cohort-level comparison of effectiveness [+W]
• Teacher experience using the software was not systematically related to changes in product effects between the first and second study year [+W]

Related Elements¶

• National experimental study of reading and mathematics software products (NCLB-mandated first student cohort)

Examples¶

-

Key Sources¶

• Larissa Campuzano, Mark Dynarski, Roberto Agodini, Kristina Rall. (2009). Effectiveness of Reading and Mathematics Software Products: Findings from Two Student Cohorts. Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. https://ies.ed.gov/ncee/