Implicit demographic conditioning inflates AES scores in some models, with Llama-70B scores rising 1.57 points over its own default, while GPT models remain stable¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? Under the implicit condition, Llama-70B essay scores inflate by +1.57 points relative to its own default (p<0.001), the largest deviation across all models and conditions. → Donya Rooein 2026 q2 i? GPT models are the most stable in AES, with no significant score shifts from explicit or implicit personas relative to human ratings. → Donya Rooein 2026 (2)

Evidence¶

Donya Rooein 2026¶

Donya Rooein, Luca Benedetto, Dirk Hovy. (2026). The Role of Implicit and Explicit Demographic Signals in Large Language Model-based Student Assessment. https://arxiv.org/abs/2609.16993 doi:10.48550/arxiv.2609.16993

q2 · i? · causal · r2

Automated essay scoring experiment with paired t-tests on 40 per-essay means under three prompting conditions. Llama-70B showed the largest implicit-condition inflation; the human mean score was 3.02±1.00. The printed value is a score deviation, not a standardized effect size.

"The Imp condition reveals the starkest effect: Llama-70B scores inflate by +1.57 points relative to its own default (p<0.001 ), the largest deviation observed across all models and conditions."

Donya Rooein 2026 (2)¶

Donya Rooein, Luca Benedetto, Dirk Hovy. (2026). The Role of Implicit and Explicit Demographic Signals in Large Language Model-based Student Assessment. https://arxiv.org/abs/2609.16993

q2 · i? · causal · r2

Same AES experiment: GPT-mini and GPT-nano showed no significant score shifts from either explicit or implicit persona conditioning relative to human ratings, while several Llama and Qwen models shifted significantly (e.g., Llama-3B +0.48, p<0.01, in the Model condition).

"In addition, GPT models are the most stable, and neither Exp nor Imp personas produce significant shifts in scores relative to human ratings."

Discussion¶

Related Claims¶

• Explicitly mentioned education and SES cues lead most models to produce less readable responses for higher-education users, showing partial readability adaptation — related
• In one reviewed study, GPT-4 automated essay scoring showed score inflation, with AI–human rating discrepancies exceeding 1.5 standard deviations in 38% of graduate-level bibliographies — related