Multidimensional latent variable models rarely support more than about four to six latent variables, limiting fine-grained SKIVE modeling¶

Claim · All claims
Evidence · 1 study · theoretical r3 · q2
Currency · newest study 2010 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2013: tool and model accuracy changes quickly
Evidence quality · No study behind this claim is a peer-reviewed article with a DOI: it rests on a source with no registered publication record (a web page, book or unpublished report). It is kept because a design page cites it; treat it as a lead to check, not an established finding.

Subclaims¶

q2 i? The literature on multidimensional latent variable models shows it is numerically very challenging to differentiate multiple latent dimensions; successfully estimated models contain at most about four to six latent variables, and highly correlated within-domain dimensions may produce unreliable scores. → Rupp 2010

Evidence¶

Rupp 2010¶

Rupp, A.A., Gushta, M., Mislevy, R.J., & Shaffer, D.W. (2010). Evidence-centered Design of Epistemic Games: Measurement Principles for Complex Learning Environments. Journal of Technology, Learning, and Assessment, 8(4). https://ejournals.bc.edu/index.php/jtla/article/view/1623

q2 · i? · theoretical · r3

The paper's evidence-model section reports, citing Haberman (2008) and Sinharay, Haberman, and Puhan (2007), that multidimensional models estimated with real data sets rarely exceed four to six latent variables and that highly correlated dimensions yield unreliable scores.

"Most latent variable models that are estimated successfully with real data sets contain between four and six latent variables at the most. Moreover, when the latent variables represent multiple latent characteristics within a single domain, they are often highly correlated and may also produce relatively unreliable scores (e.g., Haberman, 2008; Sinharay, Haberman, & Puhan, 2007)."

Discussion¶

Related Claims¶

• The second-order multidimensional IRT model for testlets is formally equivalent to the testlet model, and both are restricted bifactor models — related
• The three composite scoring methods yield differently related ability estimates, with the UIRT oral composite correlating far more with Speaking than Listening estimates — related
• Defining and quantifying reliability and measurement error is particularly challenging in epistemic games for three principal reasons — related
• Adding Markov structures for both general and specific dimensions over time yields a model that does not scale well with the number of measurement occasions — related
• Measurement models based on vaguely quantified behavioral frequency reports differ in structure from those based on numerically quantified reports — related
• Computational complexity of the graph-based MML procedure scales with the number of latent variables within a conditionally independent subset, and brute-force integration scales exponentially with dimensionality — related
• Full information MML estimation of a multidimensional IRT model with a second-order dimension also requires only two-dimensional integrals — related