A multi-armed bandit controller achieves scoring accuracy comparable to exhaustive grid search while reducing LLM calls by 78.4% and token consumption by 72.8%¶

Claim · All claims
Evidence · 1 study · causal r1 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? On IELTS Task 2 essays, the MAB framework achieved comparable QWK to grid search while reducing LLM calls by 78.4% and tokens by 72.8%. → Olga Manakina 2026

Evidence¶

Olga Manakina 2026¶

Olga Manakina, Igor Bogdanov. (2026). Learning to Grade Efficiently: A Bandit-Driven Prompt-Selection Framework for Low-Cost LLM Essay Scoring. https://arxiv.org/abs/2608.23814 doi:10.48550/arxiv.2608.23814

q2 · i? · causal · r1

Simulation-style experiment on the IELTS Task 2 subset using Google Gemini 2.5, comparing an epsilon-greedy MAB controller against exhaustive grid search. The article reports "MAB reduced total experimental costs by approximately 70%" ($0.4 vs $1.4) with comparable accuracy; Table 1 prints 78.4% call and 72.8% token reductions.

"Grid Search had consumed approximately 9 million tokens compared to MAB's 1.8 million for similar coverage. This efficiency translated directly to cost savings, as shown in Figure 7, where MAB reduced total experimental costs by approximately 70% compared to Grid Search ($0.4 versus $1.4) while maintaining comparable accuracy outcomes."

Discussion¶

Related Claims¶

• The multi-step grading recipe with calibration examples achieves the highest scoring accuracy and receives the majority of bandit arm pulls — related
• Removing detailed rubric explanations from prompts improved grading accuracy across all approaches in an ablation study — related
• Grading recipes without calibration examples perform substantially worse on MAE, though multi-step without examples ranks second on QWK — related
• An optimistic prior distribution partially mitigates MAB power loss without substantially reducing student benefits — related
• Over long horizons, MAB assignment places fewer total students in the less effective condition than a shorter uniform experiment followed by committing to one condition — related
• MAB assignment increases the Type I (false positive) error rate above the nominal alpha level — related
• MAB assignment yields higher average rewards (student outcomes) than uniform random assignment during experiments — related
• MAB (Thompson sampling) assignment reduces statistical power relative to uniform random assignment, requiring roughly doubled sample sizes — related
• Removing grid-search initialization of IRT discrimination and difficulty parameters drops scoring performance from QWK 0.747 to 0.705 — related
• The four grading recipes exhibit an accuracy-cost tradeoff, with the most accurate recipe costing roughly five times the balanced alternative — related
• MAB assignment overestimates effect sizes for normally distributed rewards, with greater overestimation for smaller true effects — related