A prior-study corpus of 84 peer tutoring sessions was coded for help type and conceptual content with kappa of 0.83 and 0.72¶

Claim · All claims
Evidence · 1 study · causal r2 · q2
Currency · newest study 2014 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Human coding of 84 tutoring sessions from a previous classroom study yielded kappa=0.83 for help type and kappa=0.72 for conceptual content, providing the training corpus for the machine classifiers. → Walker 2014

Evidence¶

Walker 2014¶

Walker, E., Rummel, N., & Koedinger, K. R. (2014). Adaptive Intelligent Support to Improve Peer Tutoring in Algebra. Int J Artif Intell Educ 24, 33–61. https://doi.org/10.1007/s40593-013-0001-9

q2 · i? · causal · r2

Corpus drawn from a previous classroom study comparing adaptive and fixed peer tutoring support, with students acting once as tutor and once as tutee. The article reports "kappa=0.83" for help type and "kappa=0.72" for conceptual content, with disagreements resolved through discussion.

"There were a total of 84 tutoring sessions from both conditions, consisting of an average of 21.77 tutor lines of dialogue per session (SD=10.25). Two raters coded tutor utterances for help type (kappa=0.83) and conceptual content (kappa=0.72)."

Discussion¶

Related Claims¶

• APTA's machine classifiers reached training-data accuracy of .81 for help type and .66 for conceptual content under 10-fold cross validation — related
• Cohen's kappa exhibits a paradox in which high observed agreement yields low kappa, making it unsuitable for pass/fail decision consistency — related