On the MathDial problem set with SymPy access, o3-mini(high) and Claude 3.5 Sonnet achieved the highest problem-solving accuracy at 90.00%¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person · recheck by 2028: tool and model accuracy changes quickly

Subclaims¶

q2 i? Model comparison on MathDial problem-solving accuracy: o3-mini(high) and Claude 3.5 Sonnet obtained the highest accuracy of 90.00%, followed by Gemini 2.0 Flash (88.67%), with GPT-4o (78.67%) and GPT-4o-mini (77.33%) lowest. → Jarosław A. Chudziak and Adam Kostka 2025

Evidence¶

Jarosław A. Chudziak and Adam Kostka 2025¶

Jarosław A. Chudziak and Adam Kostka. (2025). AI-Powered Math Tutoring: Platform for Personalized and Adaptive Education. arXiv preprint. https://arxiv.org/abs/2507.12484 doi:10.48550/arxiv.2507.12484

q2 · i? · design · r2

Model-selection evaluation testing problem-solving accuracy of various LLMs with access to the SymPy tool on the MathDial problem set. The article reports "o3-mini(high) and Claude 3.5 Sonnet obtained the highest accuracy, of 90.00%"; no effect size is printed.

"Model o3-mini(high) and Claude 3.5 Sonnet obtained the highest accuracy, of 90.00%, followed by Gemini 2.0 Flash (88.67%) with GPT-4o (78.67%) and GPT-4o-mini (77.33%) obtaining the lowest scores."

Discussion¶

Related Claims¶

• With images provided, reasoning models achieve higher majority-correct accuracy on image-Required middle-school math items than non-reasoning models — related
• In a 10-student pilot, EduGuard produced higher post-test accuracy, larger normalized learning gain, lower calibration error, and lower overreliance than GPT-4o-mini Tutor — related
• Gemini Flash 2.0 was selected for production over GPT-4o-mini and Llama 3.1 for its superior faithfulness and speed-to-cost ratio — related
• FATE benchmarks rank Gemini 2.5 Flash highest among commercial AI tutors, with actionability weakest for all models — related