Clustering of TTX activity logs aligns with instructor-assigned milestone scores and outperforms a baseline averaged over all possible groupings in both exercises¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2026 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? In both TTXs, cluster cohesion measured by RMSE against instructor milestone scores was much lower than the baseline average across all possible cluster groupings, so clustering sufficiently aligns with instructor scores and outperforms the baseline. → V. Švábenský 2026

Evidence¶

V. Švábenský 2026¶

V. Švábenský, J. Vykopal, S. Leelaluk, P. Čeleda, F. Okubo, A. Shimada. (2026). Assessment in Team Problem-Solving Exercises in Computing Education. Proceedings of the 56th IEEE Frontiers in Education Conference (FIE '26). https://arxiv.org/abs/2607.19209 doi:10.48550/arxiv.2607.19209

q2 · i? · design · r2

Quantitative comparison (RQ1) of DBSCAN-based consensus clustering outputs against instructor milestone-score vectors for 13 EXF teams and 10 PHI teams, using pairwise RMSE with the average across all Bell-number groupings as baseline; clustering RMSE was lower than baseline in both exercises (e.g., EXF 0.45 vs 0.53 baseline).

"Based on Table III, the baseline RMSE (average error across all these options) is much larger than for our clustering results. Therefore, our assessment has better internal cohesion of the clusters with respect to the manually assigned scores."

Discussion¶

Related Claims¶

• Clustering yields a small number of behaviorally distinct team clusters that instructors can use for cluster-differentiated feedback — related
• GPT-4o's rubric-guided grading of team communication disagreed with instructors at near-chance levels (55% RMSE), while GPT-5.2 improved but remained insufficiently aligned (35%) — related
• Q-matrices clustered from DKT and skip-gram item embeddings do not outperform dAFM models but do outperform a randomly initialized Q-matrix — related
• Technology-enhanced tabletop exercises delivered via IXP across 25 exercises and 743 participants showed increased student engagement and collaboration and clearer instructor visibility. — related
• Behavioral clusters recur across different student groups, which the authors attribute to shared cultural norms — related
• Milestone logic lets designers adapt TTX pacing to each team instead of delivering injects at predefined times. — related