Systematic linguistic differences exist between demographic groups in PISA text responses, including response length, lexical diversity, and out-of-vocabulary tokens¶

Claim · All claims
Evidence · 1 study · associational r2 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Girls produced more tokens than boys in both correct and incorrect responses, while boys' and non-native speakers' responses showed higher type–token ratios; correct responses contained proportionally more out-of-vocabulary tokens across groups. → Nico Andersen 2025

Evidence¶

Nico Andersen 2025¶

Nico Andersen, Julia Mang, Frank Goldhammer, Fabian Zehner. (2025). Algorithmic Fairness in Automatic Short Answer Scoring. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-025-00495-5

q2 · i? · associational · r2

Descriptive analysis of response characteristics (Table 4) across gender and home-language groups: mean types, tokens, TTR, OOV tokens, and blank responses for correct (1) and incorrect (0) responses. The pre-trained German word embedding contained 830,875 tokens.

"Girls produced more tokens than boys, in both correct and incorrect responses, while boys and non-native speakers’ responses contained a higher lexical diversity due to fewer tokens resulting in a higher ratio of unique tokens (i.e., types) to the total number of tokens (type–token ratio, TTR)."

Discussion¶

Related Claims¶

• The most predictive linguistic features differ across datasets, overlapping only in lexical diversity measures — related
• Feature operationalization can reverse the direction of the relationship with essay score, as with document length and TTR variants — related