On distractor-driven science items, instruction can first strengthen students' attraction to an alternative conception before that preference declines¶

Claim · All claims
Evidence · 1 study · associational r2 · q2 · n=1,250 students, grades 8–12
Currency · newest study 1998 · not yet checked against its sources · not reviewed by a person

On items whose distractors match common student ideas, the share of students choosing an alternative conception is not a simple decreasing function of understanding.

Subclaims¶

q2 i? For items with highly attractive distractors based on alternative conceptions, instruction appeared to strengthen support for those conceptions before the preference eventually declined, an unusual psychometric profile. → Sadler 1998

q2 i? The author reads this as support for the view that such ideas may be markers of progress toward scientific understanding rather than impediments to learning, and argues that distractor-driven instruments have a psychometric profile very different from conventional tests. → Sadler 1998

Evidence¶

Sadler 1998¶

Sadler, P. M. (1998). Psychometric models of student conceptions in science: Reconciling qualitative studies and distractor-driven assessment instruments. Journal of Research in Science Teaching, 35(3), 265–296. doi:10.1002/(SICI)1098-2736(199803)35:3<265::AID-TEA3>3.0.CO;2-P

q2 · pre/post psychometric study · i? · no effect size reported in the abstract · n=1,250 students, grades 8–12 · associational · r2

Multiple-choice astronomy items whose distractors captured alternative conceptions identified in qualitative research were given to 1,250 students in grades 8 to 12 at the start and end of their introductory astronomy courses. Psychometric modelling placed the appeal of the various ideas on a single scale and showed the rise-then-fall profile for attractive distractors, as well as the ages at which particular conceptions are most prevalent. There is no comparison group; the rise is a pattern across the ability scale and the pre/post administrations, not a manipulated effect. Coded from the published abstract; the full text was not read.

Discussion¶

A rise in the number of students choosing an alternative-conception distractor after a unit is not, on this evidence, necessarily a sign that teaching failed: it may mark learners moving from no idea to a partial one. That reading fits the knowledge-in-pieces view of misconceptions as steps in a refinement. For misconception probes it means a single pre/post comparison of distractor choices can mislead.

Related Claims¶

• Student misconceptions can be read as knowledge elements that serve in some contexts and are refined and reorganised during learning, rather than as strongly held flawed ideas that instruction must confront and replace — the theory this finding fits
• LLM-generated distractors for math multiple-choice questions are judged mathematically valid more often than plausible to students — related; distractors that track student ideas