After humanization with Undetectable AI, fewer than 4% of AI-labeled rewrites remain detectable (FNR > 96%) for Pangram and GPTZero¶

Claim · All claims
Evidence · 1 study · causal r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Post-humanization false-negative rates exceed 96% for both commercial detectors across both time windows. → Jonathan A. Karr Jr. 2026

Evidence¶

Jonathan A. Karr Jr. 2026¶

Jonathan A. Karr Jr., Grigorii Khvatskii, Ting Hua, and Nitesh V. Chawla. (2026). Why AI Detection Fails for Academic Integrity. Proceedings of the ACM AI Leadership Summit (AILS '26). https://arxiv.org/abs/2608.11256 doi:10.48550/arxiv.2608.11256

q2 · i? · causal · r2

Controlled re-scoring of all LLM rewrite variants after Undetectable AI v11 humanization with Pangram 3.2 and GPTZero. The article reports that "fewer than 4% of AI-labeled rewrites remain detectable" (FNR > 96%), with post-humanization Pangram means falling to 0.03–0.04.

"After humanization, fewer than 4% of AI-labeled rewrites remain detectable for AI detectors (FNR > 96%)."

Discussion¶

Related Claims¶

• Inter-detector agreement between Pangram and GPTZero collapses after humanization, so a single vendor flag is not a reliable consensus signal — related
• No detection threshold simultaneously keeps original flag rates, light-edit capture, and AI-pool false negatives low — related
• Independent evaluation found commercial AI-text detectors neither sufficiently accurate nor sufficiently reliable for high-stakes use — a broader claim this one bears on
• Light AI-assisted edits to human-written abstracts are flagged by commercial detectors at 64–80% (Pangram) and 38–49% (GPTZero) rates — related
• Applied to 1,163 real master's theses, Pangram flagged 529 (45.5%) as containing AI-generated content, mostly at low-to-moderate levels — related
• On fully AI-generated papers produced with ChatGPT Deep Research, Pangram achieved 65% strict accuracy while Turnitin, GPTZero and Copyleaks classified essentially all papers as false negatives — reports the opposite
• Simple paraphrasing and self-editing counter-measures collapse AI detection rates to near-random levels — possibly the same claim (merge candidate)
• AI agent activity in an LMS is indistinguishable from normal student activity, defeating detection and proctoring — related
• On hybrid and humanised AI papers, Pangram correctly identified AI content in 37 of 40 cases while GPTZero and Copyleaks performed poorly and Turnitin declined on humanised texts — reports the opposite
• A linear mixed-effects model of detection error showed significant paper type, tool, and interaction effects, with underestimation largest for fully AI-generated papers and Pangram consistently producing the smallest errors — related
• Both detectors achieved substantially lower accuracy on scientific writing than on humanities texts — related