Adaptive testing explains why about one third of calibrated Spanish math items carry calibration status 10¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2020 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Because Spanish math tests are adaptive, the distribution of student responses across the item pool is not uniform as in a linear test, which is why about 1/3 of calibrated Spanish math items have calibration status 10 (calibration sample size under 1,000). → Wang 2020

Evidence¶

Wang 2020¶

Wang, S., & Li, S. (2020). Calibration of Spanish MAP Growth math tests. Portland, OR: NWEA. https://www.nwea.org/research/publication/calibration-of-spanish-map-growth-math-tests/

q2 · i? · design · r2

Authors' explanation in the conclusion for the status 10 proportion observed in the calibration (Table 2.3 shows 1,166 of 3,955 collected items at status 10). Items are labeled status 10 when calibration sample size is less than 1,000. This is the article's own interpretation of the adaptive response distribution; no effect size is printed.

"For adaptive item calibration, due to student ability difference, the distribution of the number of student responses collected for a pool items is a uniform distribution as in a linear test. This is why about 1/3 calibrated Spanish math items have calibration status 10"

Discussion¶

Related Claims¶

• Outlier rates in Spanish item calibration are extremely low, which the authors take as evidence of very high calibration quality — related
• Including or excluding calibration status 10 items in scoring has a negligible impact on Spanish student RIT scores for monolingual and bilingual students — related
• Old (English-borrowed) and newly calibrated Spanish item RITs differ negligibly, with correlations of 0.98 and no statistically significant paired t-test differences — related
• Existing item fit statistics are derived for linear tests under assumptions of no missing responses and normal ability distributions, leaving adaptive testing contexts understudied — related
• The Spanish math test population is predominantly Hispanic (about 78%), unlike the English math test population (about 15%), with implications for Spanish norms — related