Use support-level filtering across cohorts to separate persistent from cohort-specific challenging topics¶

Strategy · All strategies
Evidence · 1 claim (1 for) · 1 study (1 associational), q3 · 0 of 1 report an effect size · 1 claim rests on one study

Description¶

A prioritization strategy in which a topic is considered persistently challenging only if it appears among the ten most difficult topics in at least two of three cohorts, with quizzes of two or more questions to ensure adequate sample size. Across the five course types, "the percent of identified topics that had appropriate support level ranged from 53% to 100%", indicating that highly challenging topics for one cohort are often but not always similarly challenging for others. This filtering step separates cohort-specific challenges from persistent ones before deeper misconception analysis.

Design Implications¶

Context¶

Requirements¶

• At least three cohorts of quiz performance data per course type
• Quizzes with two or more questions to mitigate outlier influence

Constraints¶

• Topics challenging in only one cohort are excluded even if genuinely difficult for that cohort

Target Learners¶

• medical students in online biomedical science courses

Target Learning Goals¶

• prioritizing instructional interventions on persistently difficult topics

Claims¶

• Quiz-level first-attempt performance analysis across cohorts identifies persistently challenging topics in online biomedical science courses [+W]

Related Strategies¶

• Use a cohort-wide instructor dashboard of topic-level question volume, with assistant-surfaced prerequisite review suggestions as an extension

Examples¶

-

Key Sources¶

• Parker, M. J., & Zavala-Cerna, M. G. (2025). What Don't You Understand? Using Large Language Models to Identify and Characterize Student Misconceptions About Challenging Topics. https://arxiv.org/abs/2605.00294. doi:10.48550/arxiv.2605.00294