Originality outperformed Turnitin in overall accuracy and macro-average recall on a balanced EFL-relevant dataset, but both showed limited consistency across classes¶
Claim · All claims
Evidence · 1 study · designr2·q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? Originality achieved higher macro-average accuracy (0.69 vs 0.61) and macro-average recall (0.60 vs 0.51) than Turnitin, though both produced macro-average F1-scores below 0.55. → Mohammad Hadra 2026
Evidence¶
Mohammad Hadra 2026¶
Mohammad Hadra, Karleen Cambridge and Mostefa Mesbah. (2026). Evaluating the accuracy and reliability of AI content detectors in academic contexts. International Journal for Educational Integrity. https://doi.org/10.1007/s40979-026-00213-1
q2 · i? · design · r2
Evaluation of two commercial detectors on a balanced 192-text dataset (48 texts each of EFL student, professional, AI, and hybrid authorship), with outputs mapped to Human/Hybrid/AI via established thresholds. The results section reports "Originality achieved a higher macro-average accuracy (0.69) than Turnitin (0.61)" with macro-averaging used due to moderate class imbalance.
"Originality achieved a higher macro-average accuracy (0.69) than Turnitin (0.61), as well as a higher macro-average recall (0.60 vs. 0.51). Despite this, both detectors produced macro-average F1-scores below 0.55, indicating limited consistency across all classes."
Discussion¶
Related Claims¶
- Both detectors performed poorly on hybrid human–AI texts, with Originality showing near-zero recall for the Hybrid class — related
- Both detectors achieved substantially lower accuracy on scientific writing than on humanities texts — related
- Originality showed a borderline non-significant trend toward higher accuracy on professional than EFL student writing, while Turnitin showed no significant difference — related
- Detector accuracy declined significantly as text length increased, with both detectors performing best on short texts — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
originality-outperforms-turnitin-accuracy-recall |
title |
Originality outperformed Turnitin in overall accuracy and macro-average recall on a balanced EFL-relevant dataset, but both showed limited consistency across classes |
description |
Originality outperformed Turnitin in overall accuracy and macro-average recall on a balanced EFL-relevant dataset, but both showed limited consistency across classes |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2026 |
recheck_by |
2029 |
generated |
by process:wiki-ingest on 2026-10-10 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — mohammad-hadra-2026 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.