The three fine-tuned LLMs differ significantly in response length, readability, and similarity to posts¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person · recheck by 2028: tool and model accuracy changes quickly

Subclaims¶

q2 i? A multiway ANOVA found significant differences across GPT-2, Gemma, and LLaMA in word count, Flesch-Kincaid level, and TF-IDF cosine similarity. → Liu 2025

Evidence¶

Liu 2025¶

Liu, Z., Xing, W., Jiao, X., & Li, C. (2025). Exploring Fairness and Explainability in LLM-Generated Support for Online Learning Discussion Forums. Journal of Learning Analytics, 12(3), 8–33. https://doi.org/10.18608/jla.2025.8885

q2 · i? · design · r2

Descriptive analysis of responses from the three originally fine-tuned models (Table 5): average lengths of 35.04–50.08 words, FKL of 5.24 (GPT-2) versus about 7 (Gemma, LLaMA), and similarity around 0.10–0.11 on a 200-response sample; the article reports "GPT-2 produced the most readable responses (FKL = 5.24)."

"Overall, GPT-2 produced the most readable responses (FKL = 5.24), indicating that the text is understandable by a fifth-grade student, while Gemma and LLaMA had similar readability levels (FKL = 7)."

Discussion¶

Related Claims¶

• Gemma and LLaMA produce higher-quality responses than GPT-2 by TIGERSCORE, with accuracy and comprehension gaps remaining — related
• Academic word list count, word count, and Flesch-Kincaid grade level are the most important features for predicting cognitive engagement in discussion posts — related
• In the random forest, word count was the most important readability feature while traditional readability formulas ranked near the bottom — related
• Three LLMs fine-tuned on teacher-validated feedback reach reported training losses with qualitative teacher validation — related
• Verbosity preference is associated with systematic variation in LLM response length — related
• PPB maintained comparable readability on three of four measures while increasing Flesch-Kincaid toward grade level — related
• Fine-tuned 8B LLMs generate K-2 stories with difficulty-related metric scores statistically significantly different from (mostly better than) zero-shot GPT-4o and Llama 3.3 70B — related
• Revising texts to improve readability-formula scores often produces nonexistent or insignificant comprehension gains — related