Evidence on whether initial short-answer questions produce more learning than initial multiple-choice questions is mixed, with recent studies finding little or no difference¶

Claim · All claims
Evidence · 1 study (5 entries) · review r3 · q2 · i0 negligible
Currency · newest study 2017 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? The chapter reports, citing Butler and Roediger (2007), that performance on a final short-answer test 1 month later was best after an initial short-answer test. → Karpicke 2017 q2 i? The chapter reports, citing McDaniel et al. (2007a), that in an online college course initial short-answer quizzes produced larger effects than multiple-choice quizzes on unit exams. → Karpicke 2017 (2) q2 i? The chapter reports, citing McDaniel et al. (2012), that both quiz formats outperformed rereading on unit exams but did not differ from each other. → Karpicke 2017 (3) q2 i? The chapter reports, citing McDermott et al. (2014), that in seventh grade science both quiz formats beat a no-test control, with no difference or a slight multiple-choice advantage. → Karpicke 2017 (4) q2 i0 The chapter reports that across four experiments Smith and Karpicke (2014) found an overall effect size of d = 0.07 for short-answer versus multiple-choice questioning on final retention. → Karpicke 2017 (5)

Evidence¶

Karpicke 2017¶

Karpicke, J. D. (2017). Retrieval-Based Learning: A Decade of Progress. Learning and Memory: A Comprehensive Reference, 2nd edition, Volume 2. https://doi.org/10.1016/B978-0-12-809324-5.21055-9

q2 · i? · review · r3

The chapter reports Butler and Roediger (2007), using videotaped lectures and identical question stems: "performance was best when students had initially taken a short-answer test"; the initial multiple-choice test did not beat rereading.

"On a final short-answer test 1 month later, performance was best when students had initially taken a short-answer test."

Karpicke 2017 (2)¶

Karpicke, J. D. (2017). Retrieval-Based Learning: A Decade of Progress. Learning and Memory: A Comprehensive Reference, 2nd edition, Volume 2. https://doi.org/10.1016/B978-0-12-809324-5.21055-9

q2 · i? · review · r3

The chapter reports McDaniel et al. (2007a), with Web-based quizzes and feedback in an online college course: both formats beat rereading, but "initial short-answer quizzes produced larger effects than did initial multiple-choice quizzes".

"On the unit exams, both multiple-choice and short-answer formats produced positive effects, relative to rereading items, but initial short-answer quizzes produced larger effects than did initial multiple-choice quizzes."

Karpicke 2017 (3)¶

Karpicke, J. D. (2017). Retrieval-Based Learning: A Decade of Progress. Learning and Memory: A Comprehensive Reference, 2nd edition, Volume 2. https://doi.org/10.1016/B978-0-12-809324-5.21055-9

q2 · i? · review · r3

The chapter reports two online-course experiments by McDaniel et al. (2012) with feedback: both beat reread, but "there was no difference between short- answer and multiple-choice formats"; quizzes were open book and repeatable.

"On criterial unit exams, both quiz conditions outperformed a reread control condition, but there was no difference between short- answer and multiple-choice formats."

Karpicke 2017 (4)¶

Karpicke, J. D. (2017). Retrieval-Based Learning: A Decade of Progress. Learning and Memory: A Comprehensive Reference, 2nd edition, Volume 2. https://doi.org/10.1016/B978-0-12-809324-5.21055-9

q2 · i? · review · r3

The chapter reports controlled experiments by McDermott et al. (2014) in a seventh grade science classroom, with feedback: both formats beat a no-test control, "but there were either no differences between test formats or a slight advantage of multiple-choice".

"On unit exams and end-of-semester exams, both quiz conditions outperformed a no-test control condition, but there were either no differences between test formats or a slight advantage of multiple-choice over short-answer format."

Karpicke 2017 (5)¶

Karpicke, J. D. (2017). Retrieval-Based Learning: A Decade of Progress. Learning and Memory: A Comprehensive Reference, 2nd edition, Volume 2. https://doi.org/10.1016/B978-0-12-809324-5.21055-9

q2 · i0 · review · r3

The chapter reports laboratory experiments by Smith and Karpicke (2014), including a hybrid format: "Smith and Karpicke reported an overall effect size of d ¼ 0.07" for short-answer versus multiple-choice questioning.

"Across four experiments, Smith and Karpicke reported an overall effect size of d ¼ 0.07 for the effect of short-answer versus multiple-choice questioning on final retention."

Studies bearing on this claim¶

Which practice format: short answer, multiple choice, or matching the final test? This table gathers the studies behind this claim and the 11 narrower claims that bear on it: 4 distinct studies (2 quant-synthesis, 1 review, 1 causal), q2–q4, 3 of 4 with an effect size coded. One row per study, codes as each entry records them; no effect is averaged and no verdict is drawn. Built by scripts/synthesize_claim.py from eval/synthesis/; a row that looks wrong is fixed on the claim it comes from.

Discussion¶

Related Claims¶

• Whether prompting students to construct explanations during retrieval adds benefit beyond initial free recall is mixed across studies — related
• Initial multiple-choice tests with plausible, competitive alternatives outperformed initial short-answer tests on final questions that were related to but not directly tested initially — related
• Providing feedback after initial multiple-choice tests cut lure intrusions on final short-answer tests roughly in half, with immediate and delayed feedback equally effective — related
• Short-answer (production) and multiple-choice (recognition) retrieval practice produce no statistically significant difference in undergraduates' reading comprehension on a final MCQ test — related
• Renewable assignments such as student-created quiz questions transcend classroom boundaries and can ameliorate economic injustice — related
• The benefits of retrieval practice do not depend on an exact match between initial retrieval practice conditions and the final test format — related
• Initial short-answer tests outperform initial multiple-choice tests mainly when feedback follows them; without feedback, the higher initial success of multiple-choice tests can favor multiple-choice — related
• Having more multiple-choice alternatives benefits later retention when initial retrieval success is high but hurts learning when it is low — related
• The classroom testing effect transfers to mismatched test formats, but consistent quiz–exam formats yield a significantly larger effect (g = 0.531 vs. g = 0.399). — related
• Practice test format moderates the testing effect: mixed-format (g = 0.80) and multiple-choice (g = 0.70) practice tests produced larger effects than short-answer tests (g = 0.48) — reports the opposite
• Adopting mastery goals rather than performance goals during initial learning promotes transfer — related
• Practice retrieval performance did not differ significantly between the three retrieval strategy conditions — related
• In a Terracotta study across classes, retrieval practice review outperformed restudy review on subsequent exams — related
• Students created and took peer assessments in Forms, learning through writing quiz questions and answer options — related
• Participating in peer assessment produced a statistically significant but negligible improvement (d = .115) in short-term achievement on unit quizzes in a college statistics course — related
• Retrieval-based learning research emphasizes both direct and indirect effects of retrieval on learning — related