Lower-performing groups gain relative advantage because the scorer detects incorrect responses better for them, producing more false positives¶
Claim · All claims
Evidence · 1 study · designr2·q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? The scoring model exhibited better detection rates for incorrect responses among lower-performing groups (boys and non-native speakers), which, combined with their higher prevalence of incorrect responses, produced more false positives and overestimation for these groups. → Nico Andersen 2025
Evidence¶
Nico Andersen 2025¶
Nico Andersen, Julia Mang, Frank Goldhammer, Fabian Zehner. (2025). Algorithmic Fairness in Automatic Short Answer Scoring. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-025-00495-5
q2 · i? · design · r2
Group-comparison analysis of the RoBERTa-SVM confusion matrices (Table 6): TPR was consistent across groups, but incorrect responses of lower-performing groups were better detected, yielding a higher absolute number of false positives for non-native speakers despite a slightly better FPR.
"While the true positive rate (TPR) is consistent across all groups, the results show a minor difference in the detection rate of incorrect responses. For those incorrect responses, the scoring model exhibits better detection rates for the lower performing groups (i.e., boys and non-native speakers)."
Discussion¶
Related Claims¶
- The most accurate scorer showed no gender disparity but a small significant language-background disparity favoring foreign-language speakers — related
- No consistent relationship was observed between false-positive and false-negative rates and either the question or the LLM model — related
- A detector can achieve high aggregate accuracy while producing materially different over-detection rates across author groups — related
- AI-text detectors misclassify authentic TOEFL essays by non-native English speakers at a mean false-positive rate of 61.3% across seven detectors, far more often than native-speaker essays — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
lower-performers-more-false-positives-advantage |
title |
Lower-performing groups gain relative advantage because the scorer detects incorrect responses better for them, producing more false positives |
description |
Lower-performing groups gain relative advantage because the scorer detects incorrect responses better for them, producing more false positives |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2025 |
generated |
by process:wiki-ingest on 2026-10-27 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — nico-andersen-2025 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.