Rewriting MATHia word problems for struggling readers, by human experts or LLMs, sped completion by 30% and improved mastery rate¶

Claim · All claims
Evidence · 1 study · causal r? · q2
Currency · newest study 2026 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person

Subclaims¶

q2 i? Students who received word problems rewritten by either human experts or LLMs completed math problem sets 30% more quickly than in the business-as-usual condition and improved overall mastery rate. → marcs-1

Evidence¶

marcs-1¶

Murphy, A. (2026, July). Classroom-Based experimentation in a digital learning platform. Digital Promise. https://doi.org/10.51388/20.500.12265/306

q2 · i? · causal · r?

Experiments conducted under the IES-funded MARCS collaboration with CAST, embedded in MATHia for students with reading difficulties; over 97,000 students participated across the two-year grant. The report prints the 30% completion-speed figure but no standardized effect size.

"The most significant results of this work found that students who received word problems written by either human experts or LLMs, compared with a business-as-usual condition, completed math problem sets 30% more quickly and improved overall mastery rate."

Discussion¶

Related Claims¶

• 4,446 of 9,421 MATHia word problems showed larger-than-expected error-rate gaps between less- and more-skilled readers and were flagged for potential readability concerns — related
• Partial dependence plots showed higher word count, more dependent clauses, and higher custom magnitude raised predicted flag probability, with non-linear relationships — related
• In the random forest, word count was the most important readability feature while traditional readability formulas ranked near the bottom — related
• Revising texts to improve readability-formula scores often produces nonexistent or insignificant comprehension gains — reports the opposite
• A random forest classified flagged versus non-flagged word problems with the best accuracy of five tested models (AUC = 0.75) — related
• LLM-rewritten Python error messages improved subjective evaluations but did not reliably improve objective debugging performance — related