Providing the tutor with student mastery and practice-history context improved engagement and next-item correctness¶

Claim · All claims
Evidence · 1 study (3 entries) · causal r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Providing an AFPM mastery estimate decreased tutor giving away the final answer by 55.16% ± 7.42%, increased behavioral engagement by 2.83% ± 1.73%, and decreased text complexity (reading level) by 5.59% ± 1.57%. → Udeshi 2026 q2 i? Providing AFPM levels of prerequisite skills with instructions to request review and give worked examples increased next-item correctness by 2.74% ± 2.46%. → Udeshi 2026 (2) q2 i? Providing a problem attempt history summary increased next-item correctness by 3.37% ± 3.79% and behavioral engagement by 2.49% ± 2.69%. → Udeshi 2026 (3)

Evidence¶

Udeshi 2026¶

Udeshi, T., Khazenzon, A., Khan, K., Breen, N., Corwin, R., DiGiano, C., Weatherholtz, K., Zaluski, M. (2026). Methodologies for Improving the Quality of AI Tutoring in K-12 Education. https://sites.google.com/khanacademy.org/aied2026supplementalmaterial/home doi:10.1007/978-3-032-29755-6_14

q2 · i? · causal · r2

Live experiment supplying an estimate of the user's mastery level on an AFPM scale (Unfamiliar, Attempted, Familiar, Proficient, Mastered) computed from practice history, with the tutor's only other instruction "Please tailor your tutoring style."

"Tutor giving away final answer decreased by 55.16% ± 7.42% ● Behavioral engagement increased by 2.83% ± 1.73% ● Text complexity (reading level) decreased by 5.59% ± 1.57%"

Udeshi 2026 (2)¶

Udeshi, T., Khazenzon, A., Khan, K., Breen, N., Corwin, R., DiGiano, C., Weatherholtz, K., Zaluski, M. (2026). Methodologies for Improving the Quality of AI Tutoring in K-12 Education. https://sites.google.com/khanacademy.org/aied2026supplementalmaterial/home. doi:10.1007/978-3-032-29755-6_14

q2 · i? · causal · r2

Live experiment providing the tutor with AFPM levels of all prerequisite skills and instructing it to ask students to review skills at attempted or familiar level and to provide a worked example, as illustrated in Table 2.

"Next item correctness increased by 2.74% ± 2.46%"

Udeshi 2026 (3)¶

Udeshi, T., Khazenzon, A., Khan, K., Breen, N., Corwin, R., DiGiano, C., Weatherholtz, K., Zaluski, M. (2026). Methodologies for Improving the Quality of AI Tutoring in K-12 Education. https://sites.google.com/khanacademy.org/aied2026supplementalmaterial/home. doi:10.1007/978-3-032-29755-6_14

q2 · i? · causal · r2

Live experiment providing Khanmigo a summary of the student's attempt history for the exercise, including questions attempted in the last hour with correctness and counts prior to the last hour; the tutor was only instructed to tailor its tutoring based on the additional context.

"Next-item-correctness increased by 3.37% ± 3.79% ● Behavioral engagement increased by 2.49% ± 2.69%"

Discussion¶

Related Claims¶

• Contingent scaffolding improves learning more than fixed or absent support. — the general claim this finding bears on, with the studies behind it
• Limiting verbose Math Agent guidance sharply reduced answer giveaway but decreased cognitive engagement — related
• Model selection experiments show newer models are not strict improvements, with component-specific effects on quality metrics — related
• LLMKT's predicted knowledge change curves on CoMTA are mixed across the 15 most frequent KCs, though overall they mostly resemble the power law of practice when dialogues have sufficient turns. — related
• CLST's predicted mastery levels track response correctness and move similarly for related knowledge components — related
• Most models struggle to engage with a student's prior debugging attempts even when the iteration history is provided — related
• Cumulative rapid experimentation improved next-item correctness by 10% and cognitive engagement by 14% over five months — a broader claim this one bears on
• Tutors with Tutor CoPilot access use more understanding-fostering strategies (e.g., prompting students to explain) and rely less on giving away answers than control tutors — related