Sentiment scores of LLM-generated replies differ significantly from human replies under both classifiers¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person · recheck by 2028: tool and model accuracy changes quickly

Subclaims¶

q2 i? Paired t-tests showed significant differences between sentiment scores of LLM-generated and human replies regardless of classifier. → Liu 2025

Evidence¶

Liu 2025¶

Liu, Z., Xing, W., Jiao, X., & Li, C. (2025). Exploring Fairness and Explainability in LLM-Generated Support for Online Learning Discussion Forums. Journal of Learning Analytics, 12(3), 8–33. https://doi.org/10.18608/jla.2025.8885

q2 · i? · design · r2

Paired t-tests on 8,322 pairs comparing human reply sentiment with each fine-tuned model's replies, using both BERT and SOWB classifiers; the article reports "significant differences between the sentiment scores" for all three models, with Table 6 printing means such as 4.18 (0.41) vs. 3.84 (0.51) for GPT-2 under BERT.

"It can be observed that regardless of whether BERT or SOWB was used as the sentiment classifier, there are significant differences between the sentiment scores of the responses generated by the three models and the sentiment scores of human responses."

Discussion¶

Related Claims¶

• Counterfactual fine-tuning reduces sentiment bias in LLM-generated forum replies — related
• LLM-generated forum replies are more neutral and less varied in sentiment than human replies — related
• Both fine-tuned models show regression toward the mean in score-wise bias, and error rises monotonically above score 3.0, with high scores hardest to predict — related
• LLM-generated news sentiment features did not improve ETF price-forecasting model performance in this project — related