CTML-guided educator review of AI video scripts yields statistically significant improvements across all 12 CTML principles and overall instructional validity¶

Claim · All claims
Evidence · 1 study · causal r1 · q3
Currency · no dated study recorded · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? In a within-subject study with 23 educators, the review-based approach raised mean ratings from 3.07 to 3.86 on a 5-point scale (+0.79, p<.01), with statistically significant improvements across every principle. → educator-study-1

Evidence¶

educator-study-1¶

Yearim Kim, Injun Baek, and Nojun Kwak. 2026. When Saying No Makes Better Videos: Designing Dual Gatekeeping for Pedagogically Grounded AI Content Creation. In Proceedings of Understanding and Engaging Critical Resistance to AI in Education (CHI '26 Workshop). ACM, New York, NY, USA. https://arxiv.org/abs/2608.19812 doi:10.48550/arxiv.2608.19812

q3 · i? · causal · r1

Within-subject study with 23 educators across three topics of differing cognitive demand; participants rated 13 items covering all 12 CTML principles. The article reports "The mean rating rose from 3.07 to 3.86 on a 5-point scale (+0.79, 𝑝<. 01)" with Wilcoxon signed-rank tests; no standardized effect size is printed for the overall gain.

"The review-based approach yielded statistically significant improvements across every principle (𝑝<. 05, Wilcoxon signed-rank). The mean rating rose from 3.07 to 3.86 on a 5-point scale (+0.79, 𝑝<. 01)"

Discussion¶

Related Claims¶

• Educators did not perceive the iterative CTML review process as slowing them down, rating production efficiency at 4.26/5 — related
• Automated CTML metrics show significant improvement in temporal contiguity and coherence for CTML-informed videos, while modality, redundancy, and image quality show no significant difference — related
• The pipeline feasibly produces coherent, synchronized educational videos from textbook queries, integrating retrieval, image generation, animation, and speech — related
• The CTML research base grew from 45 experimental comparisons and 7 principles (2001) to 201 comparisons and 15 principles (2021) — related