Item Response Theory as a framework for scale development¶

Theory · All theories
Evidence · 2 claims (2 for) · 1 study (1 review), q2 · 0 of 1 report an effect size · 2 claims rest on one study

Description¶

Item response theory (IRT) is presented as one of two fundamental approaches to scale development, alongside classical test theory (CTT). Unlike CTT, where observed scores of people are the focus, in IRT "the items are the focus", and person and item characteristics become independent of one another and can be placed on the same scale. IRT models (dichotomous 1PLM/2PLM/3PLM and polytomous GRM, GPCM, PCM, RSM, NRM) rest on strong assumptions of unidimensionality and local independence, and provide item and test information functions as an alternative to classical reliability. The review uses this framework to evaluate 19 Turkish scale development studies.

Design Implications¶

Context¶

Requirements¶

• IRT requires strong assumptions, principally unidimensionality of the measured construct and local independence of item responses, which must hold for inferences to be correct.

Constraints¶

• The review notes polytomous IRT models are more complex and have not yet been developed as much as dichotomous IRT models.

Target Learners¶

• Educational and psychological measurement researchers
• Scale developers

Target Learning Objectives¶

• Understanding IRT models, assumptions, and their application to scale development

Claims¶

• Irt Assumptions Often Unchecked Turkey [+M]
• Sample sizes in Turkish IRT-based scale development studies ranged from 166 to 2223, with about half below the recommended 500 [+W]

Related Theories¶

• Classical Test Theory (CTT) and its stated limitations
• Many-Facet Rasch Model (MFRM)
• Graphical model framework for deriving efficient MML estimation schemes for multidimensional IRT models
• RIT vertical equal-interval scale grounded in item response theory

Examples¶

• Use IRT rather than classical test theory when implementing repeated measurement linking designs with re-randomized groups and multiple concurrent links
• Choose CTT or MFRM for open-ended test development based on practicality and reported detail, not difficulty results
• Rasch rating-scale analysis methodology for deriving school survey measures
• Seven practical recommendations for IRT-based scale development

Key Sources¶

• Kılıç, A. F., Koyuncu, İ, & Uysal, İ. (2023). Scale development based on item response theory: A systematic review. International Journal of Psychology and Educational Studies, 10(1), 209-223. https://dx.doi.org/10.52380/ijpes.2023.10.1.982