Item parceling at the testlet level substantially improves model fit, allowing partial (Rasch, 2PL) or full (bifactor) recovery of CAT constructs¶

Claim · All claims
Evidence · 1 study · causal r2 · q2
Currency · newest study 2012 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person · overdue for a recheck since 2015: tool and model accuracy changes quickly

Subclaims¶

q2 i? At the testlet level, item parceling substantially improves model fit, and CAT constructs can be recovered partially for unidimensional models and fully for the bifactor model under the stricter fit criteria. → Shudong Wang 2012

Evidence¶

Shudong Wang 2012¶

Shudong Wang, Hong Jiao. (2012). Examine Construct Validity of Computerized Adaptive Test in K-12 Assessments. Paper presented at the annual meeting of the National Council on Measurement in Education. https://www.nwea.org/research/publication/examine-construct-validity-of-computerized-adaptive-test-in-k-12-assessments/

q2 · i? · causal · r2

Simulation results at the testlet level (Tables 7 and 8): the article reports "CAT data fit 2PL and bifactor models well based on relaxed fit criteria," with fit indices substantially improved relative to item-level data.

"Second, at the testlet level, we demonstrate that item parceling substantially improve models fit. At least for criteria set one, constructs of CAT can be recovered partially for unidimensional models (Rasch and 2PL) and fully for the multidimensional model (bifactor model)."

Discussion¶

Related Claims¶

• The study examines how CAT test design and item bank distribution affect content coverage and test efficiency in a CCSS-aligned reading comprehension test — related
• CAT data cannot be fit into factor models at the item level, making item-level construct validity investigation meaningless for CAT — reports the opposite
• A one-factor unidimensional model is the most reasonable fit for MAP computerized adaptive Reading and Mathematics tests across grades, terms, and ten states, though Michigan Mathematics misses the RMSEA criterion — related
• MCAR item and response missingness in linear tests shows no differential effect on model fit across Designs 1 to 4 — related
• Parceling over-fits models: at the testlet level, data from more complex models fit well with simpler models — related
• Ignoring testlet effects by fitting a unidimensional 2PL model produced mild shrinkage of item intercept and loading estimates toward zero relative to the bifactor model — related
• In an empirical comparison on a testlet-based English assessment test, the bifactor model is preferred over the second-order and unidimensional 2PL models by both AIC and BIC — related
• CAT development decisions influence and might threaten content alignment — related
• Under ordinal (WLSMV) estimation the bifactor model is not identifiable, with the S-1 diagnostic implicating the effectiveness-specific factor — related
• GLAT shows robust structural validity and reliability under a 2-parameter logistic IRT model — related
• Model-data fit evaluation in IRT is claimed to be crucial for both linear and adaptive tests because item response model goodness-of-fit is relevant to any test purpose — related