No detection threshold simultaneously keeps original flag rates, light-edit capture, and AI-pool false negatives low¶

Claim · All claims
Evidence · 1 study · causal r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? Across τ ∈ {0.4, 0.5, 0.6}, Pangram holds roughly 15% original flags and 80% refine flags, while raising τ from 0.4 to 0.6 slightly increases FNR on AI-labeled rewrites (10.5% to 13.8%). → Jonathan A. Karr Jr. 2026

Evidence¶

Jonathan A. Karr Jr. 2026¶

Jonathan A. Karr Jr., Grigorii Khvatskii, Ting Hua, and Nitesh V. Chawla. (2026). Why AI Detection Fails for Academic Integrity. Proceedings of the ACM AI Leadership Summit (AILS '26). https://arxiv.org/abs/2608.11256 doi:10.48550/arxiv.2608.11256

q2 · i? · causal · r2

Threshold sensitivity sweep on the 2023–2025 pre-humanization collection (Table 5 and Appendix K), reporting original flag rate, refine flag rate, and AI-pool FNR at each threshold. The article reports "Pangram holds≈15% original flags and≈80% refine flags across that band."

"No threshold𝜏∈{ 0.4, 0.5, 0.6} simultaneously keeps original flag rates, refine capture, and AI-pool FNR low (Table 5): Pangram holds≈15% original flags and≈80% refine flags across that band, while raising𝜏 from 0.4 to 0.6 slightlyincreasesFNR on AI-labeled rewrites (10.5% to 13.8%)."

Discussion¶

Related Claims¶

• After humanization with Undetectable AI, fewer than 4% of AI-labeled rewrites remain detectable (FNR > 96%) for Pangram and GPTZero — related
• Light AI-assisted edits to human-written abstracts are flagged by commercial detectors at 64–80% (Pangram) and 38–49% (GPTZero) rates — related
• Detector flag rates on 2023–2025 original abstracts vary sharply by domain, with non-STEM rates far above STEM — related
• Applied to 1,163 real master's theses, Pangram flagged 529 (45.5%) as containing AI-generated content, mostly at low-to-moderate levels — related
• On fully AI-generated papers produced with ChatGPT Deep Research, Pangram achieved 65% strict accuracy while Turnitin, GPTZero and Copyleaks classified essentially all papers as false negatives — related
• On hybrid and humanised AI papers, Pangram correctly identified AI content in 37 of 40 cases while GPTZero and Copyleaks performed poorly and Turnitin declined on humanised texts — related
• A linear mixed-effects model of detection error showed significant paper type, tool, and interaction effects, with underestimation largest for fully AI-generated papers and Pangram consistently producing the smallest errors — related
• Inter-detector agreement between Pangram and GPTZero collapses after humanization, so a single vendor flag is not a reliable consensus signal — related