Quiz-level first-attempt performance analysis across cohorts identifies persistently challenging topics in online biomedical science courses¶

Claim · All claims
Evidence · 1 study · associational r2 · q3
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? Analyzing first-attempt performance on topic-focused quizzes across three cohorts identifies topics that are consistently challenging and central to course objectives. → Parker 2025

Evidence¶

Parker 2025¶

Parker, M. J., & Zavala-Cerna, M. G. (2025). What Don't You Understand? Using Large Language Models to Identify and Characterize Student Misconceptions About Challenging Topics. https://arxiv.org/abs/2605.00294 doi:10.48550/arxiv.2605.00294

q3 · i? · associational · r2

Analysis of 9 course periods across 5 online biomedical science courses with 3,802 enrollments used first-attempt quiz scores to rank topics. "we analyzed quiz results across the 9 courses (3 cohorts) for each subject to derive the most challenging topics", filtering for topics appearing in the top 10 most difficult in at least two of three cohorts.

"we analyzed quiz results across the 9 courses (3 cohorts) for each subject to derive the most challenging topics"

Discussion¶

Related Claims¶

• Faculty corroborate data-driven identification of challenging topics as consistent with their classroom observations — a narrower finding that bears on this claim
• CTT-based quiz difficulty rankings are validated by IRT analysis with near-perfect rank correlation — related
• Classroom quizzing delivered by clickers, computer software, or paper improves student performance on classroom exams in middle school and college courses — related
• Chatbot outreach showed suggestive effects on assignment completion in Microeconomics but no effects on assignments in Government — related