Limiting verbose Math Agent guidance sharply reduced answer giveaway but decreased cognitive engagement¶

Claim · All claims
Evidence · 1 study · causal r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Focusing the Math Agent on analyzing student math steps decreased tutor giving away the final answer by 85.5% ± 5.56% and time-to-first-token latency by 7.85% ± 0.57%, increased behavioral engagement by 7.66% ± 2.25%, but decreased cognitive engagement by 18.09% ± 7.45%. → Udeshi 2026

Evidence¶

Udeshi 2026¶

Udeshi, T., Khazenzon, A., Khan, K., Breen, N., Corwin, R., DiGiano, C., Weatherholtz, K., Zaluski, M. (2026). Methodologies for Improving the Quality of AI Tutoring in K-12 Education. https://sites.google.com/khanacademy.org/aied2026supplementalmaterial/home doi:10.1007/978-3-032-29755-6_14

q2 · i? · causal · r2

Live experiment removing the Math Agent's verbose guidance section, hypothesizing it was no longer necessary with recent LLM improvements. The authors report the guidance section "previously was influencing the LLM response too much and diluting the main workflow system prompt."

"Tutor giving away final answer decreased by 85.5% ± 5.56% ● Cognitive engagement decreased by 18.09% ± 7.45% ● Time to first token latency (P50) decreased by 7.85% ± 0.57% ● Behavioral engagement increased by 7.66% ± 2.25%"

Discussion¶

Related Claims¶

• Model selection experiments show newer models are not strict improvements, with component-specific effects on quality metrics — related
• Providing the tutor with student mastery and practice-history context improved engagement and next-item correctness — related
• Reducing Math Agent output and disabling it for non-math courses reduced latency and increased behavioral engagement — related
• Prompting is a major source of AI-assisted cognitive effort: copying prompts reduces effort but not time, and verbose model responses can make AI-assisted completion slower than predicted — related
• Instances occurred where the Teacher LLM directly provided the final answer, bypassing the intended cognitive process, and can be programmatically filtered — related
• Recent developments in instructional research and instructional design models support guidance during instruction — related
• Adolescent students' stated pre-chat intentions with an AI math tutor tilt strongly toward scaffolded support rather than answer offloading — related
• Tutors with Tutor CoPilot access use more understanding-fostering strategies (e.g., prompting students to explain) and rely less on giving away answers than control tutors — related
• GPT-4 generates targeted corrective feedback for 66% of student errors, but 35% of hints are too general, incorrect, or give away the answer — related