Students answer lower-taxonomy-category questions more frequently, but only by a factor of about 2¶

Claim · All claims
Evidence · 1 study · associational r1 · q2
Currency · newest study 2013 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Across the four course datasets, questions in lower taxonomic categories tended to be answered more, by a relatively small factor (e.g. category 1 and 2 questions in Physics 1A 2011 averaged 21(4) and 18(4) answers). → Bates 2013

Evidence¶

Bates 2013¶

Bates, S. P., Galloway, R. K., Riise, J., and Homer, D. (2013). Assessing the quality of a student-generated question repository. https://arxiv.org/abs/1308.2202 doi:10.48550/arxiv.1308.2202

q2 · i? · the article prints no effect size for this finding · associational · r1

Descriptive analysis of mean answers per question by taxonomic category for the 602 evaluated questions, shown for Physics 1A in Figure 5. The article reports "category 1 and 2 questions are answered on average 21(4) and 18(4) times" and a factor-of-2 difference. No inferential test is printed.

"in questions sampled from the Physics 1A 2011 repository, category 1 and 2 questions are answered on average 21(4) and 18(4) times, respectively, where the values in parentheses give the standard error on the mean."

Discussion¶

Related Claims¶

• Students perform better overall on LOCS exam questions than on HOCS exam questions — the general claim this finding bears on, with the studies behind it
• Question cognitive-level distributions differed significantly between years for Physics 1B but not Physics 1A — related
• Student-authored physics questions mostly require application or analysis, with under 5% at the lowest recall level — related
• 75% of student-authored physics questions met combined high-quality criteria (clear, correct, plausible distractors, above recall, adequate explanation) — related
• University examination questions concentrate at the lower cognitive levels of Bloom's Taxonomy, dominated by comprehension and knowledge — related