Students whose tutors have access to Tutor CoPilot are 4 percentage points more likely to master lesson topics (pass exit tickets) in live K-12 math tutoring¶

Claim · All claims
Evidence · 3 studies (5 entries) · 2 causal r2–r3, 1 review r2 · q2–q4
Currency · newest study 2026 · checked against its sources 2026-10-17 (automated, 5 of 5 entries) · not reviewed by a person

Subclaims¶

q4 i? In a preregistered intent-to-treat analysis of the RCT, students working with treatment tutors were 4 p.p. more likely to pass exit tickets (62%→66%, p<0.01). → Rose E. Wang 2025 q4 i? Students were 2 p.p. more likely to attempt the exit ticket (p<0.1) and, conditional on attempting, 3 p.p. more likely to pass it (p<0.01). → Rose E. Wang 2025 (2) q4 i? In a 2SLS analysis of tutors who actually used the tool, students were 14 p.p. more likely to pass exit tickets (p<0.01). → Rose E. Wang 2025 (3)

q3 i? Providing human tutors with real-time LLM-generated pedagogical suggestions increased the probability of students mastering knowledge components by 4 percentage points on average, particularly for lower-performing or less-experienced tutors. → Kirk Vanacore 2026

q2 i? In a randomised controlled trial of a human–AI system in live tutoring, situational-level real-time guidance for tutors raised students' topic mastery by four percentage points overall and by nine for students of lower-rated tutors. → Mutlu Cukurova 2026

Evidence¶

Rose E. Wang 2025¶

Rose E. Wang, Ana T. Ribeiro, Carly D. Robinson, Susanna Loeb, and Dora Demszky. (2025). Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertise. arXiv:2410.03017v2. https://arxiv.org/abs/2410.03017 doi:10.48550/arxiv.2410.03017

q4 · i? · causal · r2

Preregistered intent-to-treat regression from the randomized controlled trial (900 tutors initially assigned; 4,136 sessions), with student covariates and school-by-grade fixed effects. The article reports a significant positive treatment effect: "4 percentage points (p.p.) more likely to pass their exit tickets (p< 0.01". No standardized effect size is printed.

"Students working with treatment tutors were 4 percentage points (p.p.) more likely to pass their exit tickets (p< 0.01, 62%→ 66% student passing rate from control to treatment)."

Rose E. Wang 2025 (2)¶

Rose E. Wang, Ana T. Ribeiro, Carly D. Robinson, Susanna Loeb, and Dora Demszky. (2025). Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertise. arXiv:2410.03017v2. https://arxiv.org/abs/2410.03017

q4 · i? · causal · r3

Same ITT analysis, secondary session-level measures from Table 2. Attempting the exit ticket was marginal (p<0.1); conditional passing was significant (p<0.01). Both effects are positive (2 and 3 percentage points), but the article prints no standardized effect size for either, so the magnitude is coded i?.

"We also found that students were 2 p.p. more likely to attempt the exit ticket (p< 0.1), and conditional on their attempt, they were 3 p.p. more likely to pass the exit ticket (p< 0.01)."

Rose E. Wang 2025 (3)¶

Rose E. Wang, Ana T. Ribeiro, Carly D. Robinson, Susanna Loeb, and Dora Demszky. (2025). Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertise. arXiv:2410.03017v2. https://arxiv.org/abs/2410.03017

q4 · i? · causal · r3

Two-stage least squares analysis disentangling actual use from mere access, reported in a Results-section footnote. Students of tutors who used the tool were "14 p.p. more likely to pass their exit tickets (p< 0.01". No standardized effect size is printed.

"Appendix H reports results on the tutors using Tutor CoPilot, rather than just having access to it, where students were 14 p.p. more likely to pass their exit tickets (p< 0.01)."

Kirk Vanacore 2026¶

Kirk Vanacore, Ryan S. Baker, Avery H. Closser, and Jeremy Roschelle. (2026). The Path to Conversational AI Tutors: Integrating Tutoring Best Practices and Targeted Technologies to Produce Scalable AI Agents. arXiv preprint. https://arxiv.org/abs/2602.19303 doi:10.48550/arxiv.2602.19303

q3 · i? · review · r2

The review reports this large-scale randomized controlled trial by Wang et al., a human-in-the-loop design where humans review and edit AI-generated messages. The printed gain is a positive "4 percentage points on average"; no standardized effect size is printed, so the magnitude is coded i?. The review does not name Tutor CoPilot, the population or the exit-ticket measure; it describes the outcome as mastering knowledge components, and those details come from the trial itself (Wang et al. entries above).

"In a large-scale randomized controlled trial, Wang et al. [78] found that providing human tutors with real-time, LLM-generated pedagogical suggestions increased the probability of students mastering knowledge components by 4 percentage points on average."

Mutlu Cukurova 2026¶

Mutlu Cukurova. (2026). What do you mean by 'human–AI collaboration'? Prerequisite functions and the affordances needed to achieve it. Submitted to the Handbook of AI and the Future of Education. https://orcid.org/0000-0001-5843-4854

q2 · i? · causal · r?

The chapter reports, citing Wang et al., a randomised controlled trial of a human–AI system in live tutoring in which "situational-level real-time guidance for tutors raised students' topic mastery by four percentage points overall", and by nine points for students of lower-rated tutors.

"in a randomised controlled trial of a human–AI system in live tutoring, situational-level real-time guidance for tutors raised students' topic mastery by four percentage points overall, and by nine for the students of lower-rated tutors"

Discussion¶

Related Claims¶

• Tutor Copilot No Significant Effects Secondary Outcomes
• Tutor Copilot Benefits Concentrated Among Lower Rated Tutors
• Tutors with Tutor CoPilot access use more understanding-fostering strategies (e.g., prompting students to explain) and rely less on giving away answers than control tutors — related
• Tutors perceive Tutor CoPilot as helpful for on-the-spot explanations but flag suggestions that are not grade-level appropriate and 'too smart' — related
• In one RCT, a content-rich prompt-engineered generative AI chat tutor improved performance and engagement in a Harvard physics course — related
• After training, tutors were significantly more likely to encounter pedagogical opportunities (61.1% to 68.9%) and showed higher execution quality within those opportunities (65.5% to 68.1%) — related
• Human tutoring has a medium effect over no tutoring, and in one small study students learned as well when tutors only prompted them as when tutors also explained and gave feedback — related
• Human oversight remained necessary alongside AI-based tutoring, especially for misconceptions and advanced topics — related
• GPT-generated feedback passes simulated-student helpfulness evaluations only about 35% of the time on real student data — related