Response styles as construct-irrelevant variation in Likert scale use¶

Theory · All theories
Evidence · 2 claims (2 for) · 1 study (1 associational), q1 · 0 of 1 report an effect size · 2 claims rest on one study

Description¶

The article frames response styles as differences in how respondents use the categories of a Likert scale despite having the same true score on the construct of interest. For example, respondents "may be more likely to use the extremes of the response scale independent of their true score." This framing motivates scoring surveys with IRT models that do or do not account for response styles before estimating growth.

Design Implications¶

Context¶

Requirements¶

• Survey items using Likert response categories scored on the same construct
• Longitudinal score data suitable for growth modeling

Constraints¶

• Response styles are defined relative to a respondent's true score on the construct of interest

Target Learners¶

• K-12 students completing social-emotional learning surveys

Target Learning Objectives¶

• Accurate measurement of social-emotional construct growth over time

Claims¶

• Response Styles Construct Irrelevant Bias Survey [+W]
• The effect of response styles on growth estimates varies by psychological construct, response style, and IRT model used [+W]

Related Theories¶

• Response styles as scale-usage differences independent of true score

Examples¶

• Compare growth estimates from IRT scoring models that do and do not account for response styles when evaluating longitudinal survey scores

Key Sources¶

• James Soland, Megan Kuhfeld. (2020). Do response styles affect estimates of growth on social-emotional constructs? Evidence from four years of longitudinal survey scores. Multivariate Behavioral Research. https://www.nwea.org/research/publication/do-response-styles-affect-estimates-of-growth-on-social-emotional-constructs-evidence-from-four-years-of-longitudinal-survey-scores-journal/