Larger dense generators passed smoke probes but all exceeded the +30% classroom latency limit in bake-off round 1¶
Claim · All claims
Evidence · 1 study · designr3·q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? In bake-off round 1, larger dense candidates passed all 12/12 live probes but their median completion times were 1.75–3.36 times the 8B baseline, failing the speed gate. → Sidney Shapiro and Joshua Lindemann 2026
Evidence¶
Sidney Shapiro and Joshua Lindemann 2026¶
Sidney Shapiro and Joshua Lindemann. (2026). On-Premises Multi-Course RAG Tutoring for Business Education: Hardware–Software Trade-offs in a Campus AI Tutor. arXiv. https://arxiv.org/abs/2610.02510 doi:10.48550/arxiv.2610.02510
q2 · i? · design · r3
Server-local bake-off round 1 on Node A with fixed retrieval, context length 8192, and output cap 768 tokens; the decision table reports "All passed 12/12 live probes" yet every candidate exceeded the +30% latency limit, so 8B was retained.
"All passed 12/12 live probes; median completion times were 1.75–3.36 times the 8B baseline."
Discussion¶
Related Claims¶
- Qwen 7B and Mistral Nemo 12B passed the classroom speed gate in round 2 and were retained as candidates pending faculty review — related
- Purpose-trained single-pass generation completes a slide in a median of 17 seconds and an interactive page in 59 seconds over 220k production requests, and delivers artifacts at 15–22× lower per-artifact API cost than flagship models — related
- In Quantitative AI sessions, faster AI replies were associated with more engagement, more correct practice, and larger learning gains; these associations were not significant in Verbal — related
- A structured mixture-of-experts candidate improved some corrections on fixed-evidence cases but introduced new factual and continuity regressions, so activation was held — related
- On the live single-node deployment, short-answer rounds take ≈32 s (≈$0.015/round) while multiple-choice rounds are ≈4× faster (≈7.6 s, ≈$0.005/round) because grading is deterministic — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
larger-generators-fail-classroom-speed-gate |
title |
Larger dense generators passed smoke probes but all exceeded the +30% classroom latency limit in bake-off round 1 |
description |
Larger dense generators passed smoke probes but all exceeded the +30% classroom latency limit in bake-off round 1 |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2026 |
recheck_by |
2029 |
generated |
by process:wiki-ingest on 2026-10-11 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — sidney-shapiro-and-joshua-lindemann-2026 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.