Among the three 8B LLMs, Llama 3 underperforms Apertus and Granite 3.3 on most evaluation metrics¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? Across the three 8B LLMs examined (except the baseline experiment), stories generated by Llama 3 always underperform those from the other two LLMs on most evaluation metrics. → Qian Shen 2026

Evidence¶

Qian Shen 2026¶

Qian Shen, Fanghua Cao, Min Yao, Shlok Gilda, Bonnie J. Dorr, Walter L. Leite. (2026). Children's English Reading Story Generation via Supervised Fine-Tuning of Compact LLMs with Controllable Difficulty and Safety. https://arxiv.org/abs/2605.13709 doi:10.48550/arxiv.2605.13709

q2 · i? · design · r2

Cross-model comparison in the Results section: Llama 3 stories "always underperform those from the other two LLMs" on most metrics outside the baseline, while Granite 3.3 and Apertus show no marked differences between experimental designs.

"Across the three 8B LLMs we examined (except for our baseline experiment), stories generated by Llama 3 always underperform those from the other two LLMs on most evaluation metrics. In contrast, the evaluation results of stories produced by Granite 3.3 and Apertus do not show marked differences between the various experimental designs."

Discussion¶

Related Claims¶

• Fine-tuned 8B models show suboptimal inter-story diversity, repeating names, animals, locations, and activities across narratives — related
• Model size is not the primary factor in outcome-prediction performance — related
• Under identical LoRA hyperparameters and training data, the 27B Gemma model outperforms the 70B LLaMA model on every computed essay-scoring metric — related
• Rewarded SFT produces the most stable outputs among the SFT designs, nearly free of hallucinations and story-irrelevant material — related
• Fine-tuned 8B LLMs generate K-2 stories with difficulty-related metric scores statistically significantly different from (mostly better than) zero-shot GPT-4o and Llama 3.3 70B — related
• Toxicity of stories generated by the fine-tuned 8B LLMs is very close to zero, with only a minimal number of stories posing potential safety risk — related