Some items positioned close together across booklets were also flagged as DIF, possibly due to Type I error from sampling¶

Claim · All claims
Evidence · 1 study · associational r2 · q2
Currency · newest study 2021 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2024: tool and model accuracy changes quickly

Subclaims¶

q2 i? Items placed at similar positions in different booklets were flagged as DIF in some conditions by at least one method, so not all flagged DIF can be attributed to position. → Soysal 2021

Evidence¶

Soysal 2021¶

Soysal, S., & Yilmaz Kogar, E. (2021). An Investigation of Item Position Effects by Means of IRT-Based Differential Item Functioning Methods. International Journal of Assessment Tools in Education, 8(2), 239-256. https://doi.org/10.21449/ijate.779963

q2 · i? · associational · r2

In the TEOG booklet DIF analysis, items 2 and 13 in Booklet A occupy 3rd and 14th positions in Booklet C yet were flagged; the authors state the reason "may not be based on item position" and "may have arisen due to a type 1 error caused by sampling."

"However, in the present study, the same items positioned close to each other in different booklets were also revealed to flag as DIF in some conditions (with or without purification) in at least one method"

Discussion¶

Related Claims¶

• The ICC and LDF methods identify substantially different sets of items as exhibiting DIF on a state science performance assessment — the general claim this finding bears on, with the studies behind it
• Items flagged as DIF were generally positioned at considerably different places between booklets, suggesting position-related DIF — reports the opposite
• The polytomous DFIT framework shows Type I error rates close to nominal alpha except when the number of DIF items and DIF magnitude are highest — related
• An item positioned later in one booklet was found more difficult by that booklet's test takers, attributed to fatigue effect — related
• The overall CT–PCK Survey functioned similarly across teacher characteristic groups, though some items showed raw differential item functioning above 0.64 logits — related
• Type I error rates of all three polytomous DIF indices increase as the number of DIF items and DIF magnitude increase — related