SimCalc replacement units produced statistically significant pretest-to-posttest gains in conceptual understanding with student-level effect sizes of .63, .50, and .56 in three Texas experiments¶

Claim · All claims
Evidence · 1 study · causal r2 · q4 · i2 medium
Currency · newest study 2018 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q4 i2 Across two randomized experiments and one quasi-experiment in Texas, students in SimCalc classrooms showed greater pretest-to-posttest gains than control classrooms, with student-level effect sizes of .63, .50, and .56. → Roschelle 2018

Evidence¶

Roschelle 2018¶

Roschelle, J., Tipton, E., Shechtman, N., Vahey, P. (2018). Generalizability of a Technology-Based Intervention to Enhance Conceptual Understanding in Mathematics (SimCalc Technical Report 10). Menlo Park, CA: SRI International. https://digitalpromise.dspacedirect.org/items/7a123d3c-22ec-46db-b7cf-709c3aa65583

q4 · i2 · causal · r2

Three Texas studies in 2005-06 and 2006-07 (two randomized, one quasi-experimental with same-teacher cohorts) compared 3-week SimCalc replacement units to traditional instruction using hierarchical linear modeling. The article reports "student-level effect sizes of .63, .50, and .56" favoring treatment.

"Analyses revealed statistically significant main effects of pretest to posttest gain, with student-level effect sizes of .63, .50, and .56 (Roschelle et al, 2010), with greater student learning gains for students in classrooms that used the SimCalc intervention"

Discussion¶

Related Claims¶

• SimCalc scale-up study showed significant positive impacts on curriculum-aligned conceptual items but no significant difference on state test items — related
• SimCalc treatment effects were not sensitive to school socioeconomic status or percentage of Hispanic students, and 91% of 7th-grade treatment classrooms exceeded the control-group mean gain — related
• An adapted SimCalc intervention in Florida produced strong learning gains (ES = 1.09), significantly higher posttests than predicted from the Texas control group — related
• More than 50% of participating teachers continued using the SimCalc intervention a year after the research ended, with perceived coherence and unique value predicting continued use — related
• An adapted SimCalc linear-function unit in England produced significant posttest gains versus Texas-control predictions (ES = 0.57), with no significant gain differences by poverty or prior achievement — related
• Propensity score analysis estimated a Texas generalizability index of 0.91, and post-stratification yielded a population average treatment effect of 1.47 versus a conventional estimate of 1.44 — related