Adjusting MAP Growth K–2 item difficulties improved item fit, with old infit and outfit statistics more scattered than new ones in both samples¶

Claim · All claims
Evidence · 1 study (2 entries) · associational r2 · q2
Currency · newest study 2019 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2022: tool and model accuracy changes quickly

Subclaims¶

q2 i? Old infit and outfit statistics are more scattered than new statistics, with more values larger than 2, suggesting the difficulty adjustment improved item fit. → He 2019 q2 i? Average absolute distances of new infit and outfit statistics from 1.00 are smaller than those of old statistics in both samples. → He 2019 (2)

Evidence¶

He 2019¶

He, W. (2019). MAP Growth K–2 item fit analysis study. NWEA. https://www.nwea.org/research/publication/map-growth-k-2-item-fit-analysis-study/

q2 · i? · associational · r2

Numerical results plotted in Figures 2.1–2.4 for rescored MAP Growth K–2 item responses (2010–2017, five states). The report states the old statistics "are more scattered than their new counterparts (red), with more values larger than 2 regardless of the sample."

"As shown in the figures, the old infit and outfit statistics (green) are more scattered than their new counterparts (red), with more values larger than 2 regardless of the sample. This suggests that the adjustment of item difficulties has helped with the improvement of the item fit."

He 2019 (2)¶

He, W. (2019). MAP Growth K–2 item fit analysis study. NWEA. https://www.nwea.org/research/publication/map-growth-k-2-item-fit-analysis-study/

q2 · i? · associational · r2

Summary statistics in Table 2.2: for the AllYr Reading sample, mean absolute differences from 1.00 were 0.04 (new infit) and 0.06 (new outfit) versus 0.06 and 0.09 for old statistics; the same pattern held for LatestYr and Mathematics.

"Overall, these results indicate that the absolute differences between the new statistics and 1.00 are smaller than those between the old statistics and 1.00, which serves as additional evidence that item fit has been improved with the adjusted item difficulties from the scale alignment study."

Discussion¶

Related Claims¶

• Adjusting MAP Growth K–2 item difficulties in the 2020 scale alignment study improved item fit, with the largest gains in outfit statistics for difficult Mathematics items — a narrower finding that bears on this claim
• Item fit findings were the same regardless of whether the AllYr or LatestYr response sample was used — related
• Point measure correlations were unchanged by the difficulty adjustment because the new person ability estimates are linear transformations of the old ones — related
• Mathematics items showed more variation in fit statistics than Reading items, and outfit statistics varied more than infit statistics — related
• New and old fit statistics are moderately correlated, with Pearson coefficients mostly between 0.50 and 0.79 — related
• New and old fit statistics are highly correlated after difficulty adjustment, with outfit correlations near 0.99 for Mathematics and infit correlations around 0.7 for Mathematics and mid-to-high 0.8–0.9 for Reading — related