Item position effects as an interaction between item location and test-taker performance¶

Theory · All theories
Evidence · 2 claims (2 for) · 1 study (1 associational), q2 · 0 of 1 report an effect size · 2 claims rest on one study

Description¶

Item position effects (IP effects) are defined as "the interaction between the position of a test item in a test booklet and the performance a test taker displays on the same item" (Qian, 2014, as cited). Under this account, item parameters such as difficulty or discrimination can vary according to an item's position in the booklet; classically, items near the end may be easier owing to practice or learning effects or harder owing to fatigue effects. Because IP effects are not the same for every test-taker, ignoring them may limit fair comparison and bias ability parameter estimates, threatening the validity of test score interpretations.

Design Implications¶

Context¶

Requirements¶

• Detection requires administering the same items in differing positions, e.g., via multiple booklets, and comparing item parameters or response probabilities across groups.

Constraints¶

• The article notes literature results are inconsistent: some studies find position affects performance or parameter estimates while others conclude item position does not affect examinee performance.

Target Learners¶

• Examinees of moderate to extensive length tests using multiple booklets, e.g., 8th-grade large-scale assessment takers

Target Learning Objectives¶

• Valid and invariant measurement of achievement across test forms

Claims¶

• Items flagged as DIF were generally positioned at considerably different places between booklets, suggesting position-related DIF [+W]
• An item positioned later in one booklet was found more difficult by that booklet's test takers, attributed to fatigue effect [+W]

Related Theories¶

Examples¶

• Position the same items in similar locations across different test booklets to avoid position-related DIF

Key Sources¶

• Soysal, S., & Yilmaz Kogar, E. (2021). An Investigation of Item Position Effects by Means of IRT-Based Differential Item Functioning Methods. International Journal of Assessment Tools in Education, 8(2), 239-256. https://doi.org/10.21449/ijate.779963