Test length, focal group distribution, number of DIF items, and direction of DIF had little effect on detection rates¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 1996 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 1999: tool and model accuracy changes quickly
Evidence quality · No study behind this claim is a peer-reviewed article with a DOI: it rests on an ERIC document (a report, conference paper or thesis). It is kept because a design page cites it; treat it as a lead to check, not an established finding.

Subclaims¶

q2 i? Across the simulated conditions, test length (20 vs. 40 items), impact, DIF proportion, and DIF direction showed little effect on true- and false-positive rates. → Flowers 1996

Evidence¶

Flowers 1996¶

Flowers, C. P., Oshima, C., & Raju, N. (1996). A Description and Demonstration of the Polytomous-DFIT Framework. https://eric.ed.gov/?id=ED401319

q2 · i? · design · r2

Simulation study with five replications per condition; Tables 3 and 5 report true/false positive proportions near ceiling across conditions. The authors note the high detection rate "created aceiling effect thatlimited the investigation of the influenceof factors that weremanipulated."

"impact), number of DIFitems (0%, 5%, 10%, and20%), and direction of DIF(unidirectional andbalanced-bidirectional) had little effect on the truepositive and false positivedetection rates across allconditions."

Discussion¶

Related Claims¶

• Type I error rates of all three polytomous DIF indices increase as the number of DIF items and DIF magnitude increase — related
• NCDIF Type I error rates are unaffected by focal group distribution, unlike SIBTEST and Lord's chi-square — related
• C-DIF was less stable than NC-DIF across simulated conditions — related
• The polytomous-DFIT framework effectively identified DTF and DIF in polytomously scored data under the simulated conditions — related
• DIF detection rates for all indices are higher with larger samples, equivalent distributions, fewer DIF items, greater DIF magnitude, and larger a-parameters — reports the opposite
• The polytomous DFIT framework shows Type I error rates close to nominal alpha except when the number of DIF items and DIF magnitude are highest — related
• Item discrimination moderates detection differently by index: NCDIF benefits from high a-parameters only at large DIF magnitude — related
• The chi-square significance test for NC-DIF is overly sensitive at large sample sizes, requiring an empirical cutoff value — related