Simulation and empirical comparison of scoring approaches for longitudinal survey growth estimation¶

Element · All elements
Evidence · 2 claims (2 for) · 2 studies (1 causal, 1 associational), q2 · 0 of 2 report an effect size · 2 claims rest on one study

Description¶

A methodological artifact consisting of a series of simulation and empirical studies in which scores were produced "in a single-cohort repeated measure design using sum scores as well as multiple IRT approaches" and the recovery of growth estimates from longitudinal growth models was compared across scoring sets. It serves as evidence for the relative performance of multidimensional IRT scoring over sum scores.

Design Implications¶

Context¶

Requirements¶

• Longitudinal self-report survey data with a consistent item set across time points

Constraints¶

• Findings pertain to single-cohort repeated measure designs of psychological and social-emotional constructs

Target Learners¶

• elementary and middle school students

Target Learning Goals¶

• recovering growth parameters accurately in longitudinal growth modeling of survey constructs

Claims¶

• Multidimensional IRT scores that account for latent variable covariances over time produce better recovery of growth parameters in longitudinal growth models than sum scores and other IRT approaches [+W]
• The effect of response styles on growth estimates varies by psychological construct, response style, and IRT model used [+W]

Related Elements¶

• Four years of longitudinal social-emotional survey scores analyzed under competing IRT scoring models

Examples¶

• Compare growth estimates from IRT scoring models that do and do not account for response styles when evaluating longitudinal survey scores

Key Sources¶

• Megan Kuhfeld, James Soland. (2020). Avoiding bias from sum scores in growth estimates: An examination of IRT-based approaches to scoring longitudinal survey responses. Psychological Methods. https://doi.org/10.1037/met0000367