In a real undergraduate final exam, the DOT optimized question assignment reduced the average collusion gain to 0.0073% from 19.23% under the conventional simultaneous common-question scenario.¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2021 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Applying the DOT platform to a final exam of 78 out of 85 undergraduate students reduced the average collusion gain by three orders of magnitude relative to the conventional scenario. → Mengzhou Li 2021

Evidence¶

Mengzhou Li 2021¶

Mengzhou Li, Lei Luo, Sujoy Sikdar, Navid Ibtehaj Nizam, Shan Gao, Hongming Shan, Melanie Kruger, Uwe Kruger, Hisham Mohamed, Lirong Xia and Ge Wang. (2021). Optimized collusion prevention for online exams during social distancing. npj Science of Learning 6:5. https://doi.org/10.1038/s41539-020-00083-3

q2 · i? · design · r2

Application of the authors' DOT platform to the final exam of an undergraduate imaging course on 28 April 2020, with 78 of 85 students, M1 = 40 questions from a pool of M2 = 60, competences estimated from the midterm. The exam showed "a reduction by three orders of magnitude from the conventional scenario" in average collusion gain.

"Quantitatively, the average collusion gain was reduced to 0.0073% from 19.23% (a reduction by three orders of magnitude from the conventional scenario)"

Discussion¶

Related Claims¶

• Numerical simulations over 500 instances show optimized DOT assignments reduce the mean average collusion gain by about three orders of magnitude, robust to noisy competence estimates and stable across class sizes from N = 20 to N = 500. — related
• Post-exam analyses found no evidence of significant collusion: paired tests of identical incorrect answers were below the 0.05 significance threshold at every false discovery rate, and the Wilcoxon signed-rank test on first-versus-last-20-question correctness yielded p = 0.3133. — related
• Final-exam score distributions under the DOT format did not differ significantly from the proctored midterm distributions of the same population (Kolmogorov-Smirnov p = 0.1574; equal-variance t-test p = 0.7997). — related
• Theorem 1 bounds the maximum individual collusion gain of the grouping-based anti-collusion scheme by (1 - 1/Q)/(M2 - M1 + 1), e.g. below 3.6% for M2 = 60, M1 = 40, Q = 4. — related