Self-hosted open-weight LLMs offer privacy, reproducibility, and environmental advantages over commercial LLM interfaces in educational ITSs¶

Claim · All claims
Evidence · 1 study · theoretical r? · q1
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q1 i? The authors argue that self-hosting an LLama-70b model reduces environmental impact, enhances scientific reproducibility, decreases dependency on commercial providers, and reduces privacy and data security risks. → Alina Deriyeva 2025

Evidence¶

Alina Deriyeva 2025¶

Alina Deriyeva, Jesper Dannath, and Benjamin Paaßen. (2025). SCRIPT: Implementing an Intelligent Tutoring System for Programming in a German University Context. Artificial Intelligence in Education (Springer). https://doi.org/10.1007/978-3-031-99261-2_16

q1 · i? · theoretical · r?

This is the authors' design argument (type e), not a tested result: the article offers reasons but no empirical measurement for the claimed advantages of self-hosting an LLama-70b model instead of "invoking any commercial LLM interface."

"we argue that such a setup is more suited to reduce the environmental impact of ITS operation (due to a smaller model size compared to very large LLMs), enhances scientific reproducibility (because we can exactly specify the model version we are using), and decreases dependency upon commercial LLM providers."

Discussion¶

Related Claims¶

• Under identical LoRA hyperparameters and training data, the 27B Gemma model outperforms the 70B LLaMA model on every computed essay-scoring metric — related
• Fine-tuned 8B LLMs generate K-2 stories with difficulty-related metric scores statistically significantly different from (mostly better than) zero-shot GPT-4o and Llama 3.3 70B — related
• Interacting with an eco-feedback LLM interface surfaces gaps in students' understanding of digital sustainability and supports learning about the environmental impact of everyday LLM use — related
• The small open-weights Qwen2.5-1.5B model is the weakest arm in both languages, with Spanish thematic weighted F1 of 0.31 zero-shot rising to 0.63 with few-shot — related
• In one essay-feedback system, fine-tuned models produced no usable deep-level feedback output, so only directly prompted LLMs were evaluated; the authors suggest prompting may be more effective than supervised fine-tuning for this task — related
• Under the fixed protocol, hosted LLMs produce the largest gain on Spanish thematic classification, with thematic macro F1 rising from 0.46 (TF-IDF) to roughly 0.82-0.86 for the two hosted LLMs — related
• Both fine-tuned models show regression toward the mean in score-wise bias, and error rises monotonically above score 3.0, with high scores hardest to predict — related
• A LoRA-adapted open-weight Gemma-3-27B model fine-tuned on 120 essays outperforms a fine-tuned GPT-3.5 baseline on TOEFL essay scoring across all reported metrics — related