In operational reading-test data, item parameter estimates from a standard 2PL/GPCM that ignores local dependence closely match those from the testlet model (average correlations 0.9893 and 0.9978)¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2010 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2013: tool and model accuracy changes quickly

Subclaims¶

q2 i? For six operational test forms, item discrimination and difficulty estimates were little affected by using the 2PL/GPCM model that ignored local item dependence. → Yanmei Li 2010

Evidence¶

Yanmei Li 2010¶

Yanmei Li, Shuhong Li, and Lin Wang. (2010). Application of a General Polytomous Testlet Model to the Reading Section of a Large-Scale English Language Assessment. ETS Research Report RR-10-21. http://www.ets.org/research/contact.html doi:10.1002/j.2333-8504.2010.tb02228.x

q2 · i? · design · r2

Analysis of the reading sections of six operational test forms (2005-2007 administration years 2006-2007, random sample of 2,000 examinees per form, 3 passages of 13-14 items). The report states "the RMSDs between the item parameters estimated from the two models were small" and that slightly higher discrimination parameters were found for the 2PL/GPCM.

"the item discrimination parameter (with respect to ) and difficulty parameters estimated from the two models were highly correlated, with an average correlation of 0.9893 for the discrimination parameters and an average correlation of 0.9978 for the difficulty parameters."

Discussion¶

Related Claims¶

• Ignoring testlet effects by fitting a unidimensional 2PL model produced mild shrinkage of item intercept and loading estimates toward zero relative to the bifactor model — related
• A multidimensional IRT model with simple structure fits testlet-based reading data better than both the general polytomous testlet model and the standard 2PL/GPCM — related
• The general polytomous testlet model recovers true item parameters well in simulation, with average correlations of 0.9680, 0.8608, and 0.9982 for discrimination and difficulty parameters — related
• In operational reading-test data, ignoring local dependence inflates reliability estimates, with passage-based alpha 2.3%-4.9% lower than item-based alpha across six test forms — related
• Some reviewed studies with polytomous items used dichotomous IRT models such as 2PLM or Rasch — related
• PPMC-N detects local dependence in dichotomous IRT models comparably to the Bayesian PPMC approach — related
• Parceling over-fits models: at the testlet level, data from more complex models fit well with simpler models — related
• GLAT shows robust structural validity and reliability under a 2-parameter logistic IRT model — related
• In simulation, ignoring local dependence overestimates reliability, and the overestimation grows with the size of the testlet effect (17.0%-21.8% decrease under the larger testlet effect vs. 4.8%-7.4% under the small) — related
• CTT-based quiz difficulty rankings are validated by IRT analysis with near-perfect rank correlation — related