Detection-led responses to AI in assessment face validity, fairness and trust limits¶

Claim · All claims
Evidence · 1 study · review r2 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person · recheck by 2028: tool and model accuracy changes quickly

Subclaims¶

q2 i? Detection-led responses face documented validity and fairness limits, including bias against non-native writers and notable error rates, and can corrode trust. → Kickbusch 2025

Evidence¶

Kickbusch 2025¶

Kickbusch, S., Ashford-Rowe, K., Kemp, A., Boreland, J., & Huijser, H. (2025). Beyond Detection: Redesigning Authentic Assessment in an AI-Mediated World. Education Sciences, 15(11), 1537. https://doi.org/10.3390/educsci15111537

q2 · i? · review · r2

A narrative-review assertion in which the authors, citing Liang et al. (2023) and Weber-Wulff et al. (2023), attribute these limits to detection approaches. The review reports this second-hand; no effect sizes or error-rate figures are printed in the quoted sentence.

"detection-led responses face well-documented validity and fairness limits (e.g., bias against non-native writers) and notable error rates; they can also corrode trust and distract from assessment design."

Discussion¶

Related Claims¶

• GPT detectors frequently misclassify non-native English writing as AI-generated, raising fairness concerns for AI policy — a narrower finding that bears on this claim
• AI-detection tools produce inequitable outcomes, disadvantaging non-native English speakers through false positives — a narrower finding that bears on this claim
• AI-text detectors misclassify authentic TOEFL essays by non-native English speakers at a mean false-positive rate of 61.3% across seven detectors, far more often than native-speaker essays — a narrower finding that bears on this claim
• AI detectors using perplexity disproportionately misclassify non-native English writers, and the mechanism plausibly extends to other low-perplexity writers — a narrower finding that bears on this claim
• Universities disabling AI detection cite false-positive risk at scale, and detector influence persists in individual grading, journal screening, and institutions without policy revision — related
• Detection-centered academic integrity regimes are unreliable and potentially biased, making sole reliance on AI detectors unsuitable for high-stakes decisions — a narrower finding that bears on this claim
• Humanizer sites frame AI humanization as a rational, ethically defensible response to flawed surveillance and false positives — related
• A majority of universities with faculty guidelines discourage reliance on AI detection tools due to unreliability and bias — a narrower finding that bears on this claim
• AI browsers completed Canvas quizzes and provided answer keys when tested on university assessments — a narrower finding that bears on this claim
• AI detectors are reported as linguistically biased, unreliable, and psychologically damaging in student discourse — a narrower finding that bears on this claim
• The review reports that AI detection tools disproportionately flag multilingual writers, which the authors frame as digital structural violence — a narrower finding that bears on this claim