Most reported student–LLM matches involved short snippets, and raising the minimum match length rapidly reduced coverage, making it a key review-workload control¶

Claim · All claims
Evidence · 1 study (2 entries) · associational r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? Across cohorts, the median reported overlap spanned 8–9 lines and 97.0–99.4% of cross-matches contained fewer than 30 matched lines. → Runlong Ye 2026 q2 i? At a 30-line minimum, 2.9–3.5% of 2021 student files and 10.8–16.7% of 2025 LLM attempts still had a match; at 50 lines student-file coverage was near zero and LLM-attempt coverage fell to 0.5–1.1%. → Runlong Ye 2026 (2)

Evidence¶

Runlong Ye 2026¶

Runlong Ye, Jing Fan, Angela Zavaleta Bernuy, Oscar Karnalim, Paul Denny, Juho Leinonen, and Michael Liut. (2026). Correctness, Convergence, and AI-Generated Code Detection: A Longitudinal Study of Student and Large Language Model Code in Introductory Programming. https://arxiv.org/abs/2610.00863 doi:10.48550/arxiv.2610.00863

q2 · i? · associational · r2

Distributional analysis of MOSS-reported matched-line counts for student–LLM cross-matches across the three cohorts, with docstrings retained as submitted or generated. The article reports the median and the share under 30 lines.

"Across cohorts, the median reported overlap spanned 8–9 lines, and 97.0–99.4% of cross-matches contained fewer than 30 matched lines."

Runlong Ye 2026 (2)¶

Runlong Ye, Jing Fan, Angela Zavaleta Bernuy, Oscar Karnalim, Paul Denny, Juho Leinonen, and Michael Liut. (2026). Correctness, Convergence, and AI-Generated Code Detection: A Longitudinal Study of Student and Large Language Model Code in Introductory Programming. https://arxiv.org/abs/2610.00863

q2 · i? · associational · r2

Post-processing sensitivity analysis (Figure 4) applying minimum match length filters to the same student–LLM match lists while keeping the MOSS algorithm unchanged, reporting retained shares of student files and LLM attempts by threshold.

"At 30 lines, 2.9–3.5% of 2021 historical-reference student files and 10.8–16.7% of 2025 LLM attempts still had a match. At 50 lines, student-file coverage was near zero, and LLM-attempt coverage fell to 0.5–1.1%."

Discussion¶

Related Claims¶

• Human-LLM agreement on a complex multi-label codebook falls well below human-human agreement, while LLM-LLM agreement is comparable to human-human agreement — related
• Student submissions matched the LLM-generated reference bank more often in later cohorts, with 2025 incidence 1.76–2.12 times the 2021 rate, persisting among full-pass submissions — related