LLaMA's LoRA adaptation compresses its prediction range to roughly 2.0–4.0, while Gemma maintains a broader distribution closer to the ground truth¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2026 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? LLaMA never predicts below 2.0 and assigns 85 essays a score of 4.0 but only 16 at 4.5 and 2 at 5.0, despite ground truth containing 46 and 25 essays at those levels; Gemma's prediction distribution more closely mirrors the ground truth. → John Maurice Gayed 2026

Evidence¶

John Maurice Gayed 2026¶

John Maurice Gayed. (2026). AiAWE: An Open-Source LLM Automated Writing Evaluation System Using LoRA-Adapted Instruction-Tuned Models. A PREPRINT. https://arxiv.org/abs/2606.12801 doi:10.48550/arxiv.2606.12801

q2 · i? · design · r2

Prediction frequency distribution analysis (Table 6) on the 360-essay held-out test set against the ETS ground-truth distribution. The article reports that "Gemma maintains a broader and more balanced prediction distribution that more closely mirrors the ground truth."

"LLaMA’s prediction distribution reveals a notable compression of the upper score range: it assigns 85 essays a score of 4.0 but only 16 at 4.5 and just 2 at 5.0, despite the ground truth containing 46 and 25 essays at those levels respectively."

Discussion¶

Related Claims¶

• A LoRA-adapted open-weight Gemma-3-27B model fine-tuned on 120 essays outperforms a fine-tuned GPT-3.5 baseline on TOEFL essay scoring across all reported metrics — related
• LoRA fine-tuning hyperparameters are not model-agnostic: increasing rank above 64 destroyed LLaMA's feedback generation while Gemma remained robust — related
• Under identical LoRA hyperparameters and training data, the 27B Gemma model outperforms the 70B LLaMA model on every computed essay-scoring metric — related
• Both fine-tuned models show regression toward the mean in score-wise bias, and error rises monotonically above score 3.0, with high scores hardest to predict — related
• LoRA adaptation adds only 0.6M trainable parameters on an 86.4M frozen backbone, keeping methods lightweight relative to LLM-based scoring — related
• Model size is not the primary factor in outcome-prediction performance — related
• Viewpoint-aware spatial reasoning is a persistent bottleneck with poor calibration of spatial uncertainty — related
• Fine-tuned PLMs exhibit label compression toward mid-scale QoF scores, never predicting extreme labels 1 and 7 — related
• Fine-tuned GPT-4o achieves state-of-the-art scoring of TOEFL argumentative essays with QWK 0.84 and RMSE 0.44 against official scores — related
• Fine-tuned PLMs outperform prompted LLMs in rubric-based teaching quality scoring accuracy, while LLMs better preserve score variability — related