Taking initial multiple-choice tests without feedback can lead students to later produce the incorrect lure answers they selected, even when an overall retrieval practice benefit occurs¶

Claim · All claims
Evidence · 1 study (2 entries) · review r2–r3 · q2

Subclaims¶

q2 i? The chapter reports, citing Roediger and Marsh (2005), that on a later cued recall test students tended to produce incorrect responses they had selected on the initial multiple-choice test. → Karpicke 2017 q2 i? The chapter reports, citing Marsh et al. (2009), that initial multiple-choice SAT II questions increased intrusions of incorrect items on a final short-answer test relative to no test. → Karpicke 2017 (2)

Evidence¶

Karpicke 2017¶

Karpicke, J. D. (2017). Retrieval-Based Learning: A Decade of Progress. Learning and Memory: A Comprehensive Reference, 2nd edition, Volume 2. https://doi.org/10.1016/B978-0-12-809324-5.21055-9

q2 · i? · review · r3

The chapter reports Roediger and Marsh (2005), who varied the number of alternatives: more alternatives raised initial errors, and "students tended to produce the incorrect responses that they had selected on the initial multiple-choice test".

"Importantly, on a subsequent cued recall test, students tended to produce the incorrect responses that they had selected on the initial multiple-choice test."

Karpicke 2017 (2)¶

Karpicke, J. D. (2017). Retrieval-Based Learning: A Decade of Progress. Learning and Memory: A Comprehensive Reference, 2nd edition, Volume 2. https://doi.org/10.1016/B978-0-12-809324-5.21055-9

q2 · i? · review · r2

The chapter reports Marsh et al. (2009), with college students answering SAT II questions and a final test 5 min later: there was an overall benefit, but multiple-choice "increased the likelihood that students would intrude incorrect items".

"However, taking the initial multiple-choice tests increased the likelihood that students would intrude incorrect items on the final short-answer test, relative to the no-test control condition."

Studies bearing on this claim¶

Can taking a multiple-choice test teach learners the wrong answers it shows them, and does feedback prevent that? This table gathers the studies behind this claim and the 3 narrower claims that bear on it: 2 distinct studies (1 review, 1 causal), q2–q3, 1 of 2 with an effect size coded. One row per study, codes as each entry records them; no effect is averaged and no verdict is drawn. Built by scripts/synthesize_claim.py from eval/synthesis/; a row that looks wrong is fixed on the claim it comes from.

Discussion¶

Related Claims¶

• In one experiment with 256 undergraduates, computer-based lesson versions that included practice produced significantly higher posttest scores than versions without practice — the general claim this finding bears on, with the studies behind it

• Feedback Enhances Retrieval Practice — the general claim this finding bears on, with the studies behind it

• Providing feedback after initial multiple-choice tests cut lure intrusions on final short-answer tests roughly in half, with immediate and delayed feedback equally effective — related
• Initial short-answer tests outperform initial multiple-choice tests mainly when feedback follows them; without feedback, the higher initial success of multiple-choice tests can favor multiple-choice — related
• Retrieval practice effects on mediator-cued final tests have been positive, but Coppens et al. (2016) concluded the true effect may be only about 0.10 to 0.20 — related
• In a Terracotta study across classes, retrieval practice review outperformed restudy review on subsequent exams — related
• Most incoming students do not understand that placement test performance determines which classes they can take — related
• On the live single-node deployment, short-answer rounds take ≈32 s (≈$0.015/round) while multiple-choice rounds are ≈4× faster (≈7.6 s, ≈$0.005/round) because grading is deterministic — related
• Feedback neutralizes the effect of prior study on final cued recall performance — related
• Feedback on a multiple-choice test sharply reduces lure intrusions on a later cued recall test, with immediate and delayed schedules equally effective — related
• Taking a multiple-choice test produces a testing effect on later cued recall, but testing without study or feedback can be net harmful — related
• Strategy perseveration: prior offloading experience biases later offloading choices — related
• Immediate feedback reduces multiple-choice lure production on later tests, and the overall positive effect of testing outweighs negative consequences — related
• Persistence of multiple-choice errors is driven by faulty reasoning that integrates the error into knowledge rather than by mere familiarity of lures — related
• Incorrect information learned from a multiple-choice test is applied and reasoned with on conceptually related transfer questions — related
• Multiple-choice testing increases the later production of multiple-choice lures as answers on cued recall tests, even under warning against guessing — possibly the same claim (merge candidate)
• Multiple-choice testing improves later cued recall of prose facts, a benefit that persists though reduced after a one-week delay — related
• Positive and negative testing effects occur for questions tapping higher levels of Bloom's taxonomy, and question level does not change the memorial consequences of testing — related
• Cued-recall testing can impair later memory for competitive, related but non-practiced information (retrieval-induced forgetting) — related
• Pretesting before instruction enhances learning of subsequently presented material even when pretest answers are wrong, with multiple-choice pretesting more effective than cued-recall pretesting — related
• Students struggle with the 'not' operator and logical operator precedence when evaluating conditional statements — related
• Taking a multiple-choice test boosts performance on later tests compared with nontested control conditions, including with complex SAT II materials — related
• In authentic undergraduate courses, taking more voluntary practice tests predicts higher final-exam performance, with and without corrective feedback — related
• Having more multiple-choice alternatives benefits later retention when initial retrieval success is high but hurts learning when it is low — related
• Retrieval practice mostly increases susceptibility to misinformation (retrieval-enhanced suggestibility) — related
• Contrasting orthographically similar L2 words during multiple-choice learning improves their subsequent recall in the productive direction — related