Adjusting MAP Growth K–2 item difficulties in the 2020 scale alignment study improved item fit, with the largest gains in outfit statistics for difficult Mathematics items¶

Claim · All claims
Evidence · 1 study (2 entries) · associational r2 · q2
Currency · newest study 2020 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2023: tool and model accuracy changes quickly

Subclaims¶

q2 i? New outfit statistics are closer to 1 than old outfit statistics for difficult items in both Reading and Mathematics, indicating improved model-data fit after difficulty adjustment. → He 2020 q2 i? For Mathematics items with RIT above 220, new infit statistics appear closer to 1 after adjustment, while Reading infit statistics show no noticeable change. → He 2020

Evidence¶

He 2020¶

He, W. (2020). MAP Growth K–2 item fit analysis: A follow-up study. NWEA. https://www.nwea.org/research/publication/map-growth-k-2-item-fit-analysis-study-a-follow-up-study/

q2 · i? · design · r2

Scatterplot-based numerical analysis (Figures 2.1 and 2.2) of new versus old infit and outfit statistics against item difficulty for the AllYr sample. The report states the adjustment of item difficulties "has helped with the improvement of the item fit."

"However, for Mathematics, the new infit statistics for those difficult items (i.e., item RIT>220) appear to have been brought closer to 1 with the adjusted item difficulties. The new outfit statistics are closer to 1 compared to their old counterparts for those difficult items in both content areas."

He 2020 (2)¶

He, W. (2020). MAP Growth K–2 item fit analysis: A follow-up study. NWEA. https://www.nwea.org/research/publication/map-growth-k-2-item-fit-analysis-study-a-follow-up-study/

q2 · i? · associational · r2

Summary statistics in Table 2.2 for both samples: mean absolute distances of outfit statistics from 1.0 shrink after adjustment, most markedly for Mathematics (0.25 to 0.07), which the report reads as improved model-data fit.

"the average distances from the expected value of 1.0 for the old (i.e., abs(Outfitold-1)) and new statistics (i.e., abs(Outfitnew-1)) are 0.05 and 0.04 for Reading and 0.25 and 0.07 for Mathematics."

Discussion¶

Related Claims¶

• Adjusting MAP Growth K–2 item difficulties improved item fit, with old infit and outfit statistics more scattered than new ones in both samples — a broader claim this one bears on
• New and old fit statistics are highly correlated after difficulty adjustment, with outfit correlations near 0.99 for Mathematics and infit correlations around 0.7 for Mathematics and mid-to-high 0.8–0.9 for Reading — related
• New and old fit statistics are moderately correlated, with Pearson coefficients mostly between 0.50 and 0.79 — related
• Mathematics items showed more variation in fit statistics than Reading items, and outfit statistics varied more than infit statistics — related
• Item fit findings were the same regardless of whether the AllYr or LatestYr response sample was used — related