Adaptive web-based system treatment showed a small, statistically non-significant advantage over control on elementary algebra post-test scores (Hedges' g = .12)¶

Claim · All claims
Evidence · 1 study · causal r2 · q3 · i0 negligible
Currency · newest study 2017 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i0 In an updated analysis correcting for instructor clustering and pre-test scores, treatment students scored higher than control students on the MDTP elementary algebra post-test, but the difference was not statistically significant and the effect size was small (Hedges' g = .12). → Hauk and Matlen 2017

Evidence¶

Hauk and Matlen 2017¶

Shandy Hauk, Bryan Matlen. (2017). Exploration of the Factors that Support Learning: Web-based Activity and Testing Systems in Community College Algebra. WestEd. https://eric.ed.gov/?id=ED567766

q3 · i0 · causal · r2

Updated analysis of post-semester MDTP post-test scores from the multi-site cluster randomized trial's practice semester, correcting for instructor clustering and algebra readiness pre-test scores. Treatment (adjusted M = 23.80, N = 80) exceeded control (adjusted M = 22.45, N = 147), but the difference was "not statistically different (Estimate = 0.93, p = 0.62)" with "Hedges' g = .12".

"students in the treatment condition (adjusted M = 23.80, unadjusted SD = 6.67, N = 80) performed higher on their post-test than students in the control condition (adjusted M = 22.45, unadjusted SD = 8.27, N = 147), albeit these mean scores, at about 1 point difference, were not statistically different (Estimate = 0.93, p = 0.62). The effect size for this difference was Hedges' g = .12"

Discussion¶

Learner Variables¶

• Algebraic Thinking — outcome: instruction changes it

Related Claims¶

• Using a WATS in community college elementary algebra corresponds to a statistically significant 2.59-point post-test gain (Hedges g = 0.32) controlling for pretest — reports the opposite
• An early aptitude-by-treatment interaction favoring lower-scoring treatment students was no longer statistically significant after updated analyses — related
• Treatment but not control instructors reported an increase in their ability to use technology for teaching mathematics after a practice semester, though not statistically significant — related
• The WATS efficacy estimate is qualified by high instructor attrition, and replication with a second cohort is underway — related
• Learning with intelligent tutoring systems yields a moderate, statistically significant overall advantage in achievement over non-ITS modes of instruction — a broader claim this one bears on
• Intelligent tutoring systems improve achievement over non-individualised instruction — a broader claim this one bears on