An adapted SimCalc intervention in Florida produced strong learning gains (ES = 1.09), significantly higher posttests than predicted from the Texas control group¶

Claim · All claims
Evidence · 1 study · causal r1 · q2 · i2 medium
Currency · newest study 2018 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i2 Florida 7th-grade classrooms using the adapted Texas unit showed pretest-to-posttest gains of ES = 1.09, and an HLM found posttest scores significantly higher than Texas-control-based predictions (contrast coefficient 6.96, p < .001). → Roschelle 2018

Evidence¶

Roschelle 2018¶

Roschelle, J., Tipton, E., Shechtman, N., Vahey, P. (2018). Generalizability of a Technology-Based Intervention to Enhance Conceptual Understanding in Mathematics (SimCalc Technical Report 10). Menlo Park, CA: SRI International. https://digitalpromise.dspacedirect.org/items/7a123d3c-22ec-46db-b7cf-709c3aa65583

q2 · i2 · causal · r1

Quasi-experimental comparison of 13 Florida treatment teachers (no Florida control group) to the Texas control group, with an HLM (Table 5) showing a Florida/Texas contrast of 6.96, p = 0.000. The article reports "learning gains ... were strong in the Florida classrooms using the treatment (ES=1.09)."

"As Figure 5 shows, learning gains from pretest to posttest were strong in the Florida classrooms using the treatment (ES=1.09)."

Discussion¶

Related Claims¶

• SimCalc treatment effects were not sensitive to school socioeconomic status or percentage of Hispanic students, and 91% of 7th-grade treatment classrooms exceeded the control-group mean gain — related
• An adapted SimCalc linear-function unit in England produced significant posttest gains versus Texas-control predictions (ES = 0.57), with no significant gain differences by poverty or prior achievement — related
• SimCalc replacement units produced statistically significant pretest-to-posttest gains in conceptual understanding with student-level effect sizes of .63, .50, and .56 in three Texas experiments — related
• Propensity score analysis estimated a Texas generalizability index of 0.91, and post-stratification yielded a population average treatment effect of 1.47 versus a conventional estimate of 1.44 — related
• More than 50% of participating teachers continued using the SimCalc intervention a year after the research ended, with perceived coherence and unique value predicting continued use — related