AI-content detection tools achieve only marginally higher success than human reviewers, around 80% versus 78.4%¶

Claim · All claims
Evidence · 1 study · theoretical r3 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person · recheck by 2028: tool and model accuracy changes quickly

Subclaims¶

q2 i? Machine detection of AI-generated text succeeds only marginally more often than human review, at around 80% versus 78.4%. → Sandra Leaton Gray 2025

Evidence¶

Sandra Leaton Gray 2025¶

Sandra Leaton Gray, Dominic Edsall, Dimitris Parapadakis. (2025). AI-Based Digital Cheating At University, and the Case for New Ethical Pedagogies. Journal of Academic Ethics. https://doi.org/10.1007/s10805-025-09642-y

q2 · i? · theoretical · r3

The article reports, citing Wahle et al. (2022), that detection tools show "only a marginally higher success rate than human reviewers". The printed rates are 80% for machines and 78.4% for humans; no effect size or significance test is printed.

"Studies show that these tools have only a marginally higher success rate than human reviewers, with detection rates around 80% for machines compared to 78.4% for humans (Wahle et al., 2022)."

Discussion¶

Related Claims¶

• Independent evaluation found commercial AI-text detectors neither sufficiently accurate nor sufficiently reliable for high-stakes use — a broader claim this one bears on
• In a paired exploratory comparison, extensive machine rewriting by a commercial humanizer materially degraded the manuscript while changing the detection score only slightly — related
• In a documented exploratory audit, one human-authored manuscript received materially different classifications from five commercial detectors, spanning 0% human to Human Generated — related
• Universities disabling AI detection cite false-positive risk at scale, and detector influence persists in individual grading, journal screening, and institutions without policy revision — related
• A detector can achieve high aggregate accuracy while producing materially different over-detection rates across author groups — related
• GPT detectors frequently misclassify non-native English writing as AI-generated, raising fairness concerns for AI policy — related
• AI-text detectors misclassify authentic TOEFL essays by non-native English speakers at a mean false-positive rate of 61.3% across seven detectors, far more often than native-speaker essays — related
• AI-detection tools produce inequitable outcomes, disadvantaging non-native English speakers through false positives — related
• AI detectors are reported as linguistically biased, unreliable, and psychologically damaging in student discourse — related
• Detection-centered academic integrity regimes are unreliable and potentially biased, making sole reliance on AI detectors unsuitable for high-stakes decisions — related
• Blinded review identified 68% of ChatGPT-generated abstracts as chatbot-produced while 14% of authentic abstracts were misclassified as chatbot-generated — a narrower finding that bears on this claim
• OpenAI's GPT-2 Output Detector distinguished real student essays from ChatGPT-generated essays on the same subjects in a December 2022 trial — related
• Human markers detect purchased contract-cheating assignments at low rates, though priming and authorship-analysis software improve detection — related
• AI detectors using perplexity disproportionately misclassify non-native English writers, and the mechanism plausibly extends to other low-perplexity writers — related
• AI-generated text can pass as human-authored in up to 80% of cases, making reliable differentiation a significant challenge — related