Tutor training performance predicts real-life tutoring transcript scores, with a one SD increase in lesson performance associated with a 0.25 SD increase in transcript scores¶
Claim · All claims
Evidence · 1 study · causalr1·q3
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q3 i? Across 405 tutor-lesson pairs with transcript data, mixed-effects models showed one SD increase in overall lesson performance was associated with a 0.25 SD increase in transcript scores (p<.001). → Thomas 2026
Evidence¶
Thomas 2026¶
Thomas, D. R., Kamikazi, M. C. A., Brandt, C., Borchers, C., & Koedinger, K. R. (2026). AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice. https://arxiv.org/abs/2606.18617 doi:10.48550/arxiv.2606.18617
q3 · i? · causal · r1
Mixed-effects analysis of 86 tutors' lesson scores against LLM-scored real-life transcripts, with random effects for repeated measures within tutors. The article reports "a 0.25 SD increase in transcript scores (p<.001)"; no standardized effect size beyond the 0.25 SD association is printed.
"Across tutor–lesson pairs with available transcript data where tutors had the opportunity to demonstrate skills (N = 405), mixed effects models indicated that one SD increase in overall lesson performance was associated with a 0.25 SD increase in transcript scores (p<.001)."
Discussion¶
Related Claims¶
- Repeated opportunities to apply tutor moves did not by themselves improve execution: opportunity count, lesson completion, and their interaction showed no significant effects on successful execution — related
- Interrupted time series analysis found tutor quality improved gradually over time (β=0.01, p=.022) with no immediate level change or slope change at the training intervention — related
- After training, tutors were significantly more likely to encounter pedagogical opportunities (61.1% to 68.9%) and showed higher execution quality within those opportunities (65.5% to 68.1%) — related
- Performance during the virtual pipetting lesson predicts real-life transfer test performance — possibly the same claim (merge candidate)
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
training-performance-predicts-transcript-scores-025sd |
title |
Tutor training performance predicts real-life tutoring transcript scores, with a one SD increase in lesson performance associated with a 0.25 SD increase in transcript scores |
description |
Tutor training performance predicts real-life tutoring transcript scores, with a one SD increase in lesson performance associated with a 0.25 SD increase in transcript scores |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2026 |
recheck_by |
2029 |
generated |
by process:wiki-ingest on 2026-10-10 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — thomas-2026 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.