Joint IRT models including collateral information (raw score, gender, ethnicity) fit better than item-only base models for both tests¶

Claim · All claims
Evidence · 1 study (2 entries) · associational r2 · q2
Currency · newest study 2011 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2014: tool and model accuracy changes quickly

Subclaims¶

q2 i? For the NS34 math test, likelihood-ratio tests showed models with raw score, raw score plus gender, and all CI variables fit better than the item-only base model (p < 0.001). → Wang 2011 q2 i? For the LI52 reading test, models with raw score and raw score plus gender fit better than the base model, but the gender-only model did not (p > 0.25). → Wang 2011 (2)

Evidence¶

Wang 2011¶

Wang, S., & Jiao, H. (2011). Incorporating Person Covariates and Response Times as Collateral Information to Improve Person and Item Parameter Estimations. Paper presented at the annual meeting of the National Council on Measurement in Education, New Orleans, LA. https://www.nwea.org/research/publication/incorporating-person-covariates-and-response-times-as-collateral-information-to-improve-person-and-item-parameter-estimations/

q2 · i? · associational · r2

Real-data analysis of 2009 MPG administrations comparing nested joint GLMM models via quasi-likelihood LR tests (Table 2). Adding raw score, gender and ethnicity to the item-only model gave G1 = -676.2 for model 2, "-733.5" for model 5, all p < 0.001.

"The results for NS34 show that G1 indicates that alternative hypotheses (H1) are true for model 2, 4, and 5 and the same is true for G2."

Wang 2011 (2)¶

Wang, S., & Jiao, H. (2011). Incorporating Person Covariates and Response Times as Collateral Information to Improve Person and Item Parameter Estimations. Paper presented at the annual meeting of the National Council on Measurement in Education, New Orleans, LA. https://www.nwea.org/research/publication/incorporating-person-covariates-and-response-times-as-collateral-information-to-improve-person-and-item-parameter-estimations/

q2 · i? · associational · r2

Same LR-test analysis for the reading test (Table 2): the gender-only model 8 had G1 = -1.8 with p > 0.25, so gender alone did not improve fit for LI52, while raw-score models did (G1 = -61.9, p < 0.001).

"The results for LI52 show that G1 indicates that alternative hypotheses (H1) are true for model 7, 8, and 9 and the same is true for G2 except for model 8."

Discussion¶

Related Claims¶

• Adding collateral information has little effect on ability and item parameter estimates but more effect on speed parameter estimates — related
• Scores from a different subject test can serve as collateral information: mathematics scores can improve reading item parameter estimation and vice versa — related