Items with low-SES ecologically relevant content show Differential Item Functioning to the disadvantage of low-SES students more often than neutral items¶

Claim · All claims
Evidence · 1 study · associational r2 · q3 · i2 medium
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i2 The odds of low-SES students answering correctly, given average math performance, are 0.91 for ecologically relevant items versus 1.02 (word problems) and 1.06 (notation) for neutral items; controlling for item features, the effect has Cohen's d = 0.70. → Marjolein Muskens 2024

Evidence¶

Marjolein Muskens 2024¶

Marjolein Muskens, Willem E. Frankenhuiss & Lex Borghans. (2024). Math items about real-world content lower test-scores of students from families with low socioeconomic status. npj Science of Learning. https://doi.org/10.1038/s41539-024-00228-8

q3 · i2 · associational · r2

Mantel-Haenszel DIF analyses of 161 TIMSS items comparing relevant-content items with neutral word problems and mathematical-notation items. A follow-up linear regression on DIF-odds controlling for item features reports "b = −0.09, t(160) =−2.55, p = 0.012, Cohen’s d = 0.70", described by the authors as a medium effect size.

"The odds for low-SES students to respond correctly, in comparison to high-SES students and given their average math performance, is signi ficantly lower (0.91) in math items with low-SESecologically relevant content than in the other items (1.02 and 1.06) (see Table3)."

Discussion¶

Learner Variables¶

• Domain Context — moderator: an instructional effect differs with it

Related Claims¶

• Low-SES students have a lower chance of correctly answering math items with money, food, or social-interaction content than expected given their average math performance — possibly the same claim (merge candidate)
• At item level, money and social-interaction content are significantly related to lower odds of correct responding by low-SES students, but food content is not when compared with neutral word problems — a narrower finding that bears on this claim
• The disadvantage from ecologically relevant content is driven by money and social-interaction items in both grades, but food items show the effect only in grade 4 — a narrower finding that bears on this claim
• The expected performance advantage of ecologically relevant content for low-SES students, predicted by the hidden talents approach, was not found; the opposite pattern emerged — a broader claim this one bears on
• Contrary to the hidden-talents hypothesis, low-SES students scored 18% lower on mathematics items of real-world importance — possibly the same claim (merge candidate)