Higher-scored learner answers tend to be longer, and crowdsourced answers tend to be shorter than original classroom data¶

Claim · All claims
Evidence · 1 study · associational r1 · q2
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Across all datasets, answers with higher content scores tend to be longer than lower-scored answers, and crowdsourced answers tend to be shorter than the original data. → Andrea Horbach 2024

Evidence¶

Andrea Horbach 2024¶

Andrea Horbach, Joey Pehlke, Ronja Laarmann-Quante, Yuning Ding. (2024). Crosslingual Content Scoring in Five Languages Using Machine-Translation and Multilingual Transformer Models. Int J Artif Intell Educ. https://doi.org/10.1007/s40593-023-00370-1

q2 · i? · associational · r1

Dataset analysis of answer length in characters across the six datasets, plotted per score and per prompt (Fig. 3), including a DeepL-translated comparison to factor out language. The article reports the length-score tendency and that English crowd-worker answers are shorter than original ASAP answers.

"We observe that across all datasets, better answers, i.e. answers with a higher score, tend to be longer than answers with a lower score. We also see differences between prompts, with prompt 2 eliciting the longest answers."

Discussion¶

Learner Variables¶

• Creativity — predictor: learners who differ on it differ in outcomes

Related Claims¶

• Part of the crosslingual performance drop is attributable to differences between learner populations and datasets, not only language — possibly the same claim (merge candidate)
• Shorter codebooks and shorter data excerpts yield higher initial LLM coding accuracy, and a smaller specialization gap between codebook and excerpt helps — related
• Machine translation of training and test data does not degrade monolingual content scoring performance and can normalize learner errors — related