Reviewer free-text comments identified synthetic voice, slide–narration reveal timing, and slide layout as the main remaining quality issues¶

Claim · All claims
Evidence · 1 study (2 entries) · design r2 · q2
Currency · newest study 2026 · checked against its sources 2026-10-22 (automated, 2 of 2 entries) · not reviewed by a person

Subclaims¶

q2 i? Across 235 free-text comments on 73 of 92 videos, slide visual quality, slide layout, and voice drew the most remarks (31, 30, and 27 comments), including digit-by-digit number reading and narration preceding slide display. → Romain Puech 2026

Evidence¶

Romain Puech 2026¶

Romain Puech, Dewang Kumar Agarwal, Antonio Santamaría Escobar, Dimitris Bertsimas. (2026). Bespoke: Generating MOOC-Quality Industry-Personalized Lecture Videos at Scale. arXiv preprint. https://arxiv.org/abs/2609.26540 doi:10.48550/arxiv.2609.26540

q2 · i? · design · r2

Qualitative grouping of reviewer free-text comments from the 92-video expert evaluation. The article reports 235 comments on 73 videos, with slide visual quality, layout, and voice the most remarked categories, and quotes complaints about pronunciation, reveal lag, and unused space.

"Reviewers left 235 free-text comments on 73 of 92 videos. Slide visual quality, slide layout, and voice drew the most remarks (31, 30, and 27 comments)."

Romain Puech 2026 (2)¶

Romain Puech, Dewang Kumar Agarwal, Antonio Santamaría Escobar, Dimitris Bertsimas. (2026). Bespoke: Generating MOOC-Quality Industry-Personalized Lecture Videos at Scale. arXiv preprint. https://arxiv.org/abs/2609.26540

q2 · i? · design · r2

Voice-related reviewer comments from the same qualitative analysis: reviewers flagged digit-by-digit reading of two-digit numbers, missing pauses, and a playback speed that sounded too fast, though several still judged the voice usable.

"Several videos read two-digit numbers digit by digit (“30” as “three zero, ” “60” as “six zero”), in at least six cases “repeated across the entire video” and “very distracting. ”"

Discussion¶

Related Claims¶

• Domain-matched experts judged 87% of Bespoke-generated lecture videos at or above the rubric midpoint corresponding to a standard MOOC lecture's quality (mean G = 3.42 of 5) — related
• Across rubric dimensions, content scored highest (A = 4.03) while production scored lowest (D = 3.41), driven by synthetic voice quality (D1 = 3.16) — related
• In early prototype testing, AI-generated visuals were the main barrier to real-world readiness, rated lower than text and narration — related
• A deterministic slide-image override converts a 0/9 corpus-grounding failure into 9/10 successful slide matches on the same topic — related
• Among 6,699 LLM-judge deduction flags, visual delivery is the largest deficiency category (39%) and learner adaptation the second (27%), while content problems are rarer (16%) — related
• Linguist review of draft items revealed communication problems with terminology and imprecisely reasoned attitude statements — related
• Learners valued AI revision podcasts for portability and 'dead time' use but identified absent pauses and AI voice monotony as design barriers — related