Scale drift across years and grades as a measurement challenge arising from changing instruction, materials, and standards¶

Theory · All theories
Evidence · 4 claims (3 for, 1 mixed) · 3 studies (2 associational, 1 theoretical), q1–q2 · 0 of 3 report an effect size · 4 claims rest on one study

Description¶

The article frames scale drift as the measurement-stability problem that arises when educational measurement scales must remain interpretable across years and grades. It states that "The development of measurement scales for use across years and grades in educational settings provides unique challenges", because "instructional approaches, instructional materials, and content standards all change periodically". The study addresses this by examining long-standing Rasch scales from both a micro view and a macro view.

Design Implications¶

Context¶

Requirements¶

• Scales intended for use across years and grades in educational settings, where comparability of scores over time is required

Constraints¶

• The challenge arises because instructional approaches, instructional materials, and content standards all change periodically

Target Learners¶

• K-12 students assessed with cross-grade educational measurement scales

Target Learning Objectives¶

• Accurate measurement and interpretation of student achievement and growth across grades and years

Claims¶

• Long Term Study Rasch Scale Stability Almost 40 Years [+W]
• Both item-level and test-level analyses indicate the MAP Growth measurement scales are remaining stable [+W]
• Meaningful longitudinal linking depends on the construct not changing across measurement occasions [~W]
• Average drift in MAP Growth item difficulty estimates is negligible across subjects, with original-new correlations above 0.99 [+W]

Related Theories¶

Examples¶

-

Key Sources¶

• A large-scale, long-term study of scale drift: The micro view and the macro view. (2016). Presentation paper, NWEA. https://www.nwea.org/research/publication/a-large-scale-long-term-study-of-scale-drift-the-micro-view-and-the-macro-view/