Post-exam analyses found no evidence of significant collusion: paired tests of identical incorrect answers were below the 0.05 significance threshold at every false discovery rate, and the Wilcoxon signed-rank test on first-versus-last-20-question correctness yielded p = 0.3133.¶

Claim · All claims
Evidence · 1 study (2 entries) · design r2 · q2
Currency · newest study 2021 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? The empirical evidence did not support an abnormal number of student pairs giving identical incorrect answers. → Mengzhou Li 2021 q2 i? The difference in average numbers of correct answers to the first and last 20 questions was not statistically significant (Wilcoxon signed-rank p = 0.3133). → Mengzhou Li 2021 (2)

Evidence¶

Mengzhou Li 2021¶

Mengzhou Li, Lei Luo, Sujoy Sikdar, Navid Ibtehaj Nizam, Shan Gao, Hongming Shan, Melanie Kruger, Uwe Kruger, Hisham Mohamed, Lirong Xia and Ge Wang. (2021). Optimized collusion prevention for online exams during social distancing. npj Science of Learning 6:5. https://doi.org/10.1038/s41539-020-00083-3

q2 · i? · design · r2

Post-exam analysis of the 78 students' answers testing whether pairs of students gave the same incorrect answer more often than the 1/4 chance rate expected under independence; all q values fell below the 0.05 significance level (Fig. 2d).

"The results of the hypothesis for testing aspect (i) confirm that the corresponding values for each false discovery rate is below the significance of 0.05 (Fig. 2d). Therefore, the empirical evidence does not support that there was an abnormal number of student pairs who consistently gave identical incorrect answers"

Mengzhou Li 2021 (2)¶

Mengzhou Li, Lei Luo, Sujoy Sikdar, Navid Ibtehaj Nizam, Shan Gao, Hongming Shan, Melanie Kruger, Uwe Kruger, Hisham Mohamed, Lirong Xia and Ge Wang. (2021). Optimized collusion prevention for online exams during social distancing. npj Science of Learning 6:5. https://doi.org/10.1038/s41539-020-00083-3

q2 · i? · design · r2

Test of whether correctness increased in the latter half of the exam, where collusion would be more likely, based on the 78 students' answers; the null of identical averages for the first and last 20 questions could not be rejected.

"we utilized the non-parametric Wilcoxon signed-rank test for paired observations49, which yielded a p value of 0.3133. Based on the evidence, we cannot reject the hypothesis that the average numbers of correct answers to the first and the last 20 questions are identical."

Discussion¶

Related Claims¶

• In a real undergraduate final exam, the DOT optimized question assignment reduced the average collusion gain to 0.0073% from 19.23% under the conventional simultaneous common-question scenario. — related
• Guidance, immediate feedback, and system-directed control showed unadjusted moderator signals that did not survive multiplicity adjustment, making them hypothesis-generating only — related
• A post-exam survey found the DOT-format online exam well received: 76.9% of students rated time sufficiency 3 or above and 80.8% rated interface convenience 3 or above on 5-point scales. — related
• Numerical simulations over 500 instances show optimized DOT assignments reduce the mean average collusion gain by about three orders of magnitude, robust to noisy competence estimates and stable across class sizes from N = 20 to N = 500. — related