The math discrepancy between NWEA and ERS recovery estimates is largely driven by differences in their samples of districts¶

Claim · All claims
Evidence · 1 study · associational r2 · q2
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? When restricted to approximately 1,150 districts common to both samples with matched inclusion rules, NWEA and ERS math recovery estimates are very similar (.033 vs .037 SDs). → Dan Dewey 2024

Evidence¶

Dan Dewey 2024¶

Dan Dewey, Erin Fahle, Tom Kane, Megan Kuhfeld, Sean F. Reardon. (2024). Reconciling Recent Evidence on Academic Recovery in the US from 2022 to 2023. https://www.nwea.org/research/publication/reconciling-recent-evidence-on-academic-recovery-in-the-us-from-2022-to-2023/

q2 · i? · associational · r2

Common-sample comparison applying the ERS methodology to NWEA districts and restricting both to the same districts tested in 2019, 2022, and 2023 with at least 80% tested. Estimates converge (".033 compared to .037 SDs"), implicating sample differences.

"In math, both studies yield very similar estimates of recovery in the set of common districts (.033 compared to .037 SDs). This implies that the difference between the two reports in math results is largely driven by differences in their samples of districts."

Discussion¶

Related Claims¶

• In the full samples, NWEA data show far smaller math recovery (.015 SDs) and negative ELA change (-.008 SDs) than ERS (.045 and .019 SDs) — related
• Sensitivity analyses restricting to students assessed in all 15 terms reproduced the full-sample between-group comparisons — related
• After excluding Illinois and Ohio, ELA scores showed essentially no recovery from 2022 to 2023 in the remaining 24 ERS states — related