Comparative studies report that AI detection tools do not reliably distinguish human-written from AI-generated text¶

Claim · All claims
Evidence · 1 study · theoretical r3 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? A comparative study of fourteen commercially available AI detection tools concluded that no tool reliably distinguished human-written from AI-generated text across all conditions, with false positives a significant problem. → Craig Wright 2026

Evidence¶

Craig Wright 2026¶

Craig Wright. (2026). Transcription is not generation: distinguishing non-generative AI tool use from academic misconduct in higher education assessment. International Journal for Educational Integrity. https://doi.org/10.1007/s40979-026-00234-w

q2 · i? · theoretical · r3

The paper reports, citing Weber-Wulff et al. (2023), a comparative study of fourteen commercially available AI detection tools which 'conclude that no tool tested reliably distinguished human-written from AI-generated text across all conditions'. The paper adopts this conclusion only to the extent supported in the cited source.

"Weber-Wulff et al. (2023), in a comparative study of fourteen commercially available AI detection tools published in this journal, conclude that no tool tested reliably distinguished human-written from AI-generated text across all conditions, and that false positives were a significant problem across all tools tested."

Discussion¶

Related Claims¶

• Independent evaluation found commercial AI-text detectors neither sufficiently accurate nor sufficiently reliable for high-stakes use — possibly the same claim (merge candidate)
• AI-generated text can pass as human-authored in up to 80% of cases, making reliable differentiation a significant challenge — a broader claim this one bears on
• In a documented exploratory audit, one human-authored manuscript received materially different classifications from five commercial detectors, spanning 0% human to Human Generated — a narrower finding that bears on this claim
• All four detection tools classified 100% of fully human-written papers as true negatives, with false positives nearly absent — reports the opposite
• AI text detectors can return high AI-detection scores for human-written academic text, raising fairness concerns about authenticity evaluation — a narrower finding that bears on this claim
• Educators struggled to reliably differentiate student-written from GenAI-generated reflections — related
• In a human Turing test, participants could not reliably distinguish BEAGLE traces from real student data (52.8% accuracy; d′ = 0.15 within the ±0.3 equivalence bound, pTOST = 0.038) — related
• Admissions officers can often discriminate AI-written from human-written essays, achieving an AUC of 0.70, well above chance but below commercial detectors — related