13%–15% of MAP Growth items per subject were flagged as significantly drifted by the Robust Z procedure, with no clear pattern among flagged items¶

Claim · All claims
Evidence · 1 study · associational r2 · q2
Currency · newest study 2021 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2024: tool and model accuracy changes quickly

Subclaims¶

q2 i? Under a two-sided Robust Z test at the 0.1 significance level, 13%–15% of items in each subject were flagged as significantly drifted, and flagged items showed no clear pattern across the scale. → Wei He 2021

Evidence¶

Wei He 2021¶

Wei He, NWEA Psychometric Solutions. (2021). An Investigation of Item Parameter Invariance Using Focused Calibration Samples for MAP Growth. NWEA. https://www.nwea.org/research/publication/an-investigation-of-item-parameter-invariance-using-focused-calibration-samples-for-map-growth/

q2 · i? · associational · r2

Study 1 applied the Robust Z method (Huynh & Rawls, 2011) to detect item parameter drift among on-grade recalibrated items (Table 3.3). The report states "13%–15% of items in each subject were found by the Robust Z statistics to be significantly drifted away from their underlying scales." Flagged items' average difficulty differences resembled unflagged items'.

"13%–15% of items in each subject were found by the Robust Z statistics to be significantly drifted away from their underlying scales."

Discussion¶

Related Claims¶

• Flagged items showed no clear pattern by original difficulty, spreading across the entire scale like unflagged items — related
• MAP tests show metric invariance across grades for all studied tests except Michigan Mathematics tests — related
• Robust Z procedure flagged between 4.2% and 6.9% of MAP Growth items as unstable across subjects — reports the opposite
• Both item-level and test-level analyses indicate the MAP Growth measurement scales are remaining stable — related