Teaching Monster Challenge benchmark¶
Product or Programme · All products and programmes
Evidence · 2 claims (2 for) · 1 study (1 design),q2· 1 of 1 report an effect size · 2 claims rest on one study
Description¶
A benchmark released by Lin et al. containing persona-conditioned instructional-video generation items, evaluation rubrics, scoring traces, and human judgments for assessing end-to-end systems.
Components¶
Claims¶
- Systems adapt to the specified learner persona in ways human raters can perceive, scoring well above chance in persona identification [+M]
- The LLM-judge separates weak submissions but its ranking of the strongest systems has almost no agreement with crowd ranking (Spearman ρ = −0.17) [+M]
Related Products and Programmes¶
Key Sources¶
- Lin, Yi-Cheng; Guo, Yu-Kai; Chen, Szu-Chi; Feng, Bo-Han; Hsu, Yun-Man; Hsieh, Hsiang; Lin, Yu-Jung; Wu, Yue-Ling; Dong, Jia-Kai; Cheng, An-Yu; Huang, Yu-Han; Ieong, Lok-Lam; Chen, Kuan-Yu; Tchouang, Ming-Douo; Sun, Shao-Hua; Lin, Che; Ding, Jian-Jiun; Lee, Hung-yi. (2026). Findings of the First Teaching Monster Challenge: A Benchmark of Pedagogical Content Knowledge in AI Agents. arXiv. https://arxiv.org/abs/2608.08852 doi:10.48550/arxiv.2608.08852
Page metadata
| Field | Value |
|---|---|
type |
product |
id |
teaching-monster-challenge-benchmark |
title |
Teaching Monster Challenge benchmark |
description |
A benchmark released by Lin et al. |
status |
review |
product_kind |
dataset |
generated |
by process:settle-candidates on 2026-10-12 |
trust |
unverified — no one has confirmed this page's content |
Frontmatter is the page's machine-readable record — see the Schema & Guide.