The design strained on later proof-heavy topics, and students asked for better artifacts rather than fewer tests¶

Claim · All claims
Evidence · 1 study (2 entries) · design r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Free-text complaints concentrated on reductions, NP-completeness, PSPACE, and an overloaded week, described as too terse. → Jin-Guo Liu 2026 q2 i? Only 4 of 18 students selected fewer tests as a top improvement; 11 of 18 wanted more worked examples and 11 a practice-problem bank. → Jin-Guo Liu 2026 (2)

Evidence¶

Jin-Guo Liu 2026¶

Jin-Guo Liu, Shang-Qi Lu, Xin-Ran Shi, Long-Li Zheng, Wei Wang. (2026). Test-Driven, AI-Assisted Learning: Replacing Lectures with Weekly Closed-Book Tests. https://arxiv.org/abs/2606.23315 doi:10.48550/arxiv.2606.23315

q2 · i? · design · r2

Free-text survey comments from the end-of-term survey (N=18), analyzed qualitatively. Complaints "concentrate on later proof-heavy topics"; the authors link this to cognitive-load and proof-scaffolding work. No effect sizes printed.

"Free-text complaints concentrate on later proof-heavy topics: reductions, NP-completeness, PSPACE, and an overloaded week. Students described these as "too terse" and asked for more figures and examples,"

Jin-Guo Liu 2026 (2)¶

Jin-Guo Liu, Shang-Qi Lu, Xin-Ran Shi, Long-Li Zheng, Wei Wang. (2026). Test-Driven, AI-Assisted Learning: Replacing Lectures with Weekly Closed-Book Tests. https://arxiv.org/abs/2606.23315

q2 · i? · design · r2

Improvement-ranking items from the same survey (N=18). "11 of 18 for more worked examples" and 11 for a practice-problem bank; the authors conclude the next iteration should improve artifacts before weakening the weekly gate.

"Only 4 of 18 selected fewer tests as a top improvement. The strongest signals were 11 of 18 for more worked examples, 11 for a practice-problem bank, 9 for an AI-workflow guide, and 7 for short videos."

Discussion¶

Related Claims¶

• Per-topic performance varied: NP-completeness scored highest ROUGE-1 F1 (0.1285), sorting algorithms highest pedagogical quality (0.8250), and recurrence relations lowest pedagogical quality (0.6629) — related
• BLEU-4 scored zero on all 179 mathematical-proof responses, which the authors interpret as a property of n-gram metrics rather than system failure — related
• Students in a 13-week lecture-free course reported accepting the weekly closed-book gate and rated the format as deeper and faster than lectures — related
• Students ranked the self-paced AI-study block and the weekly closed-book test as the components they would miss most, treating tests as useful accountability — related
• The weekly closed-book gate was attainable: 80.1% of recorded student-test observations reached the 100-point credit cap — related