IF-AT exam scores correlate with constructed-response quiz scores (r = 0.67), supporting validity¶

Claim · All claims
Evidence · 1 study · design r2 · q2 · i2 medium
Currency · newest study 2013 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2016: tool and model accuracy changes quickly

Subclaims¶

q2 i2 Among 47 students who completed course materials and wrote the final exam, cumulative IF-AT examination scores correlated r = 0.67 with scores on bi-weekly constructed-response quizzes, which the author takes as evidence of exam validity. → Slepkov 2013

Evidence¶

Slepkov 2013¶

Slepkov, A. D. (2013). Integrated Testlets and the Immediate Feedback Assessment Technique. https://arxiv.org/abs/1308.4365 doi:10.48550/arxiv.1308.4365

q2 · i2 · design · r2

Correlational validity check comparing cumulative IF-AT exam scores against five bi-weekly constructed-response quizzes graded by recitation leaders with limited role in creating the IF-AT exams. The author reports "the correlation ( r = 0 . 67)" and notes it is in line with validated MC tests.

"With 47 participants who both completed course materials and wrote the final exam, the correlation ( r = 0 . 67) is sufficiently clear to establish the validity of this particular exam as a proper assessment of student knowledge"

Discussion¶

Related Claims¶

• Polychotomously scored IF-AT physics exam items show excellent discrimination, with a mean item-total correlation of r′ = 0.41 ± 0.13 across 45 items — related
• Partial credit accounts for only a small proportion of IF-AT exam scores, though weaker students depend on it more — related
• IRAT scores correlate low-to-moderately and significantly with pharmacy calculations lecture assessment scores — related