SimCalc scale-up study showed significant positive impacts on curriculum-aligned conceptual items but no significant difference on state test items¶

Claim · All claims
Evidence · 1 study · review r2 · q3
Currency · newest study 2021 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? In the Scaling Up SimCalc study, students showed significant positive impacts on assessment items aligned to the environment's conceptual skills, while performance on state test items showed no significant difference. → Roschelle 2021

Evidence¶

Roschelle 2021¶

Roschelle, J., Mazziotti, C. & Means, B. (2021). Scaling Up Design of Inquiry Environments. In C. Chinn & R.G. Duncan et al (Eds.), International Handbook on Learning and Inquiry. Routledge. ISBN 9781138922600. https://isbnsearch.org/isbn/9781138922600 doi:10.4324/9781315685779-7

q3 · i? · review · r2

The chapter reports the Scaling Up SimCalc randomized controlled trial (Roschelle, Shechtman, et al., 2010) as an example that proximal, curriculum-aligned measures show stronger impacts than distal accountability measures. No effect size is printed in the chapter.

"researchers found significant positive impacts for assessment items related to the conceptual skills emphasized in the SimCalc mathematical inquiry environment, but no significant difference in students’ performance on items chosen from the relevant state test."

Discussion¶

Related Claims¶

• SimCalc replacement units produced statistically significant pretest-to-posttest gains in conceptual understanding with student-level effect sizes of .63, .50, and .56 in three Texas experiments — related
• More than 50% of participating teachers continued using the SimCalc intervention a year after the research ended, with perceived coherence and unique value predicting continued use — related
• State tests under NCLB measured little higher-order thinking, with only 2 percent of math items qualifying — related
• SimCalc treatment effects were not sensitive to school socioeconomic status or percentage of Hispanic students, and 91% of 7th-grade treatment classrooms exceeded the control-group mean gain — related
• An adapted SimCalc linear-function unit in England produced significant posttest gains versus Texas-control predictions (ES = 0.57), with no significant gain differences by poverty or prior achievement — related