Use DFIT indices with empirically established cutoffs and iterative linking to screen polytomous tests for DIF and DTF¶

Strategy · All strategies
Evidence · 3 claims (3 for) · 2 studies (2 causal), q2 · 0 of 2 report an effect size · 3 claims rest on one study

Description¶

The article outlines a practical workflow for test developers: estimate item parameters with PARSCALE, link the Reference Group metric to the Focal Group metric with a two-stage iterative linking procedure, compute C-DIF and NC-DIF, and apply an empirically established cutoff value (.016 in this study) to flag items. The authors suggest that "a test developercould examine the DTF,then determine which itemneeds to be eliminated based on its C-DIFvalue and its.overallcontribution to DTF."

Design Implications¶

Context¶

Requirements¶

• Item parameter estimation and equating/linking of group metrics before index computation
• An empirically or statistically established critical value for the DIF indices

Constraints¶

• The empirical cutoff used here was optimal only for this study's conditions; critical values at various alpha levels and with different IRT models still need to be established

Target Learners¶

• Test developers and psychometricians working with polytomously scored assessments

Target Learning Goals¶

• Identifying and removing differentially functioning items to reduce differential test functioning

Affordances¶

Claims¶

• An iterative two-stage linking procedure improves identification of DIF items [+W]
• The chi-square significance test for NC-DIF is overly sensitive at large sample sizes, requiring an empirical cutoff value [+W]
• Item discrimination moderates detection differently by index: NCDIF benefits from high a-parameters only at large DIF magnitude [+W]

Related Strategies¶

• Use a decision template weighing assumption fit, DIF uniformity, simultaneity, significance testing, and ease of use when choosing a DIF method for polytomous performance assessments

Examples¶

-

Key Sources¶

• Flowers, C. P., Oshima, C., & Raju, N. (1996). A Description and Demonstration of the Polytomous-DFIT Framework. https://eric.ed.gov/?id=ED401319