Gemini Flash 2.0 was selected for production over GPT-4o-mini and Llama 3.1 for its superior faithfulness and speed-to-cost ratio¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Across both disciplines, Gemini Flash 2.0 was chosen for production due to its superior faithfulness and speed-to-cost ratio compared to GPT-4o-mini and Llama 3.1. → Anna Ostrowska 2026

Evidence¶

Anna Ostrowska 2026¶

Anna Ostrowska, Michał Kukla, Gabriela Majstrak, Jan Opala, Sebastian Pergała, Jan Skwarek, Anna Wróblewska. (2026). From Surface Learning to Deep Understanding: A Grounded AI Tutoring System for Moodle. https://tinyurl.com/mrytc2dr doi:10.24963/ijcai.2026/984

q2 · i? · design · r2

LLM comparative testing conducted during the system-wide RAG optimization. The authors state Gemini Flash 2.0 was chosen for production for its "superior faithfulness and speed-to-cost ratio" against the two named alternatives.

"Across both disciplines, Gemini Flash 2.0 was chosen for production due to its superior faithfulness and speed-to-cost ratio compared to GPT-4o-mini and Llama 3.1"

Discussion¶

Related Claims¶

• On the MathDial problem set with SymPy access, o3-mini(high) and Claude 3.5 Sonnet achieved the highest problem-solving accuracy at 90.00% — related
• Model selection experiments show newer models are not strict improvements, with component-specific effects on quality metrics — related
• GPT-4o-mini natural language feedback was overwhelmingly helpful and never harmful in quality annotation — related
• Optimal RAG configuration differs by discipline: STEM subjects perform best with 512-character chunks and temperature 0.3 (faithfulness 0.97), while humanities require 1000-character chunks and temperature 0.1 (context recall 1.00, faithfulness 0.98) — related
• Running the Socrates playground and grader incurred low cost, with even gpt-4o consuming only $169.90 — related