A multidimensional IRT model with simple structure fits testlet-based reading data better than both the general polytomous testlet model and the standard 2PL/GPCM¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2010 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2013: tool and model accuracy changes quickly

Subclaims¶

q2 i? For all six real data sets, the likelihood ratio test favored the MIRT-SS model over the 2PL/GPCM, and AIC and BIC favored MIRT-SS over both alternatives except for Data D. → Yanmei Li 2010

Evidence¶

Yanmei Li 2010¶

Yanmei Li, Shuhong Li, and Lin Wang. (2010). Application of a General Polytomous Testlet Model to the Reading Section of a Large-Scale English Language Assessment. ETS Research Report RR-10-21. http://www.ets.org/research/contact.html doi:10.1002/j.2333-8504.2010.tb02228.x

q2 · i? · design · r2

Model-fit comparison across the six operational reading data sets using the likelihood ratio test, AIC, and BIC. The report states that "For all six datasets, G2 was significant, indicating that the MIRT-SS fit the data better than the 2PL/GPCM model", and that the estimated latent traits were highly correlated, ranging from 0.87 to 0.93.

"both AIC and BIC indicated that the MIRT-SS model fit the data better than the 2PL/GPCM and the general testlet model for all data sets except Data D, for which the fit of the three models was similar."

Discussion¶

Related Claims¶

• In operational reading-test data, item parameter estimates from a standard 2PL/GPCM that ignores local dependence closely match those from the testlet model (average correlations 0.9893 and 0.9978) — related
• The second-order multidimensional IRT model for testlets is formally equivalent to the testlet model, and both are restricted bifactor models — related
• In an empirical comparison on a testlet-based English assessment test, the bifactor model is preferred over the second-order and unidimensional 2PL models by both AIC and BIC — related
• Ignoring testlet effects by fitting a unidimensional 2PL model produced mild shrinkage of item intercept and loading estimates toward zero relative to the bifactor model — related
• The proportionality restrictions that the testlet model imposes on the bifactor model were implausible in the applied dataset, as specific-to-general loading quotients varied far from constant within testlets — related
• In operational reading-test data, ignoring local dependence inflates reliability estimates, with passage-based alpha 2.3%-4.9% lower than item-based alpha across six test forms — related
• Most Turkish IRT-based scale development studies did not check the unidimensionality or local independence assumptions — related
• Some reviewed studies with polytomous items used dichotomous IRT models such as 2PLM or Rasch — related
• GLAT shows robust structural validity and reliability under a 2-parameter logistic IRT model — related
• PPMC-N detects local dependence in dichotomous IRT models comparably to the Bayesian PPMC approach — related
• The assessment violates IRT assumptions of unidimensionality and local independence, which the authors identify as a likely cause of the method discrepancy — related
• The general polytomous testlet model recovers true item parameters well in simulation, with average correlations of 0.9680, 0.8608, and 0.9982 for discrimination and difficulty parameters — related
• Bifactor-(S-1) models of teacher Big Five personality fit student and teacher data better than single-domain or three-facets models — related