FATE benchmarks rank Gemini 2.5 Flash highest among commercial AI tutors, with actionability weakest for all models¶

Claim · All claims
Evidence · 1 study (2 entries) · design r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? Gemini achieved the highest overall average accuracy (82.88%), leading in Mistake Identification, Mistake Location, and Actionability, while DeepSeek performed best on Providing Guidance. → Tahmid Al Hannan 2026 q2 i? Across all four commercial models, Actionability was the weakest evaluation category. → Tahmid Al Hannan 2026 (2)

Evidence¶

Tahmid Al Hannan 2026¶

Tahmid Al Hannan, Diego Garcia, Alex Njoroge, Suha Al Juboori, Tarek Sakakini. (2026). Knowledge Distillation for Automated AI Tutor Evaluation. https://arxiv.org/abs/2607.10647 doi:10.48550/arxiv.2607.10647

q2 · i? · design · r2

FATE evaluation of tutor responses generated by Claude Sonnet 4.6, ChatGPT 5.5 Instant, Gemini 2.5 Flash, and DeepSeek V4 Flash on BEA 2025 conversation histories (Table 3). Gemini's average accuracy was 82.88% versus Claude 74.00%.

"Gemini achieved the highest overall average ac- curacy, leading in Mistake Identification, Mistake Location, and Actionability, while DeepSeek per- formed best on Providing Guidance."

Tahmid Al Hannan 2026 (2)¶

Tahmid Al Hannan, Diego Garcia, Alex Njoroge, Suha Al Juboori, Tarek Sakakini. (2026). Knowledge Distillation for Automated AI Tutor Evaluation. https://arxiv.org/abs/2607.10647

q2 · i? · design · r2

Same FATE benchmark of commercial tutors (Table 3): the article reports "Actionability remained the weakest category" for all models, with per-model actionability accuracies ranging from 46.50 (Claude) to 63.50 (Gemini).

"Across all models, Actionability remained the weakest category, indicating that providing concrete next-step guidance continues to be a challenge for current AI tutors."

Discussion¶

Related Claims¶

• Claude Opus 4.8 has the highest overall score (0.803) and is the only model ranking in the top three on all three stages, yet does not saturate situated tutoring (0.773) or workflows (0.704) — related
• An LLM-based question-specificity classifier reached .830 knowledge and .829 scaffold accuracy without fine-tuning — related
• One student group compared outputs across three LLMs and judged Gemini more accurate overall, though it still failed fine-grained distinctions — related
• On the MathDial problem set with SymPy access, o3-mini(high) and Claude 3.5 Sonnet achieved the highest problem-solving accuracy at 90.00% — related
• Gemini-3-flash achieved the highest rubric-item accuracy, ranging from 89% on Course 2 to over 99% on Course 1 — related
• Educational safety performance varies widely across LLMs, with GPT-5.6-Luna the safest and Qwen3-8B among the weakest under educational adversarial probing — related
• GPT models showed lower accuracy with more rubric-related errors (28% vs. 13% for Gemini 3 Flash) and near parity between false positives and negatives — related
• Gemini-3.1-Pro-Preview is the strongest MLLM on SVLAT, exceeding the human mean, while the three open-source models remain below the human baseline — related
• The author reports that prompt phrasing and the specific foundational model had secondary impact on final material quality compared with the structure of the workflow itself. — related