Interrupted time series analysis found tutor quality improved gradually over time (β=0.01, p=.022) with no immediate level change or slope change at the training intervention¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r1 · q3
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? An interrupted time series analysis of average transcript score showed a small but significant linear trend over time (β=0.01, p=.022), with no immediate level change at the intervention point (post: p=.246) and no slope change (time×post: p=.479). → Thomas 2026 q3 i? A logistic ITS model of opportunity likelihood showed a significant positive pre-training trend (OR=1.07, p<.001), a non-significant negative level change (post OR=0.63, p=.111), and a significantly attenuated post-training slope (time×post OR=0.99, p=.028). → Thomas 2026 (2)

Evidence¶

Thomas 2026¶

Thomas, D. R., Kamikazi, M. C. A., Brandt, C., Borchers, C., & Koedinger, K. R. (2026). AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice. https://arxiv.org/abs/2606.18617 doi:10.48550/arxiv.2606.18617

q3 · i? · causal · r1

Interrupted time series analysis of average transcript scores across the three-phase within-subjects design. Variation was largely attributable to stable differences between tutors (ICC=.27), with fixed effects explaining little variance (marginal R²=.01).

"An interrupted time series analysis of average transcript score showed a small but significant linear trend over time (β= 0.01,p=.022), indicating gradual improvement in tutor quality independent of the training intervention."

Thomas 2026 (2)¶

Thomas, D. R., Kamikazi, M. C. A., Brandt, C., Borchers, C., & Koedinger, K. R. (2026). AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice. https://arxiv.org/abs/2606.18617. doi:10.48550/arxiv.2606.18617

q3 · i? · causal · r1

Logistic mixed-effects ITS model of opportunity likelihood, with substantial heterogeneity across tutors and skills remaining (ICC=.35). The post-training slope was significantly attenuated (time×post OR=0.99, p=.028).

"A logistic mixed-effects interrupted time series model predicting the likelihood of observing an opportunity to apply a target tutoring skill revealed a significant positive pre-training trend (odds ratio, OR = 1.07 per unit time,p<.001)"

Discussion¶

Related Claims¶

• Reformed elementary schools made significant test-score improvements versus matched comparison schools, with gains growing gradually over four years — related
• Tutoring benefits both tutors and tutees — related
• Repeated opportunities to apply tutor moves did not by themselves improve execution: opportunity count, lesson completion, and their interaction showed no significant effects on successful execution — related
• After training, tutors were significantly more likely to encounter pedagogical opportunities (61.1% to 68.9%) and showed higher execution quality within those opportunities (65.5% to 68.1%) — related
• Tutor training performance predicts real-life tutoring transcript scores, with a one SD increase in lesson performance associated with a 0.25 SD increase in transcript scores — related