Differential student test effort may bias achievement gap estimates¶

Claim · All claims
Evidence · 1 study · associational r? · q1
Currency · newest study 2019 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2022: tool and model accuracy changes quickly

Subclaims¶

q1 i? Achievement gap estimates may be biased by differential student test effort, making test effort a threat to the validity of this policy metric. → James Soland 2019

Evidence¶

James Soland 2019¶

James Soland. (2019). Are achievement gap estimates biased by differential student test effort? NWEA Research. https://www.nwea.org/research/publication/are-achievement-gap-estimates-biased-by-differential-student-test-effort/ doi:10.1177/016146811812001202

q1 · i? · associational · r?

This is the brief's framing of its research question, given as the title of the associated journal article: whether "achievement gap estimates biased by differential student test effort" and the putting of "an important policy metric to the test." The supplied text prints no bias estimate or test statistic, so this records the concern as posed, not a measured bias.

"Are achievement gap estimates biased by differential student test effort? Putting an important policy metric to the test."

Discussion¶

Related Claims¶

• Basic conclusions about male-female achievement gaps and changes in gaps as students progress through school may change when models account for test effort — a narrower finding that bears on this claim
• Differences in rapid guessing between subgroups generally do not impact substantive interpretations of achievement gaps — reports the opposite
• Test effort differs substantially by student subgroup, with males rapidly guessing nearly twice as often as females in later grades and Black students rapidly guessing more often than White students — a narrower finding that bears on this claim
• Student test effort differs substantially across student gender and racial subgroups — a narrower finding that bears on this claim
• The article investigates whether achievement gap estimates are sensitive to test motivation, framing the question as achievement gap versus engagement gap — a narrower finding that bears on this claim
• One measurement article argues that low test-taking motivation biases an individual student's achievement test score downward — related