In the fine-tuning experiments, increasing base model size up to 32B parameters improved performance, and larger LLMs consistently outperformed smaller MLMs¶
Claim · All claims
Evidence · 1 study · causalr?·q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? Performance improved with base model size up to 32B parameters, and larger LLMs consistently outperformed smaller-sized MLMs, differing from prior LCP findings where smaller MLMs performed comparably to larger LLMs. → Adam Nohejl 2026
Evidence¶
Adam Nohejl 2026¶
Adam Nohejl, Xuanxin Wu, Yusuke Ide, Maria Angelica Riera Machin, Yi-Ning Chang, Hitomi Yanaka. (2026). Sakura at BEA 2026 Shared Task 1: What Makes Vocabulary Difficult? https://arxiv.org/abs/2605.14257 doi:10.48550/arxiv.2605.14257
q2 · i? · causal · r?
Conclusion-level summary of the fine-tuning experiments (Section 5.1, Table 2; Appendix E), where individual LLMs up to 32B (GLM-4-32B, Qwen2.5-32B, Ministral-3-14B) and MLMs were compared. The authors hypothesize the LLM advantage is due to the more complex nature of vocabulary difficulty prediction compared to LCP.
"increases in base model size up to 32B parameters resulted in performance improvements"
Discussion¶
Related Claims¶
- CDPK performance scales with model size, with a sharp Pareto-frontier drop-off below around 8B parameters — related
- The domain-specialized 32-billion-parameter model surpasses significantly larger general-purpose models such as Qwen3-235B-A22B-Thinking (86.63%) and Qwen2.5-72B-Instruct (78.60%) on the CDPK Benchmark — related
- Fine-tuning LLMs and MLMs with cross-entropy loss using soft targets outperforms standard discretization and regression-head approaches on vocabulary difficulty prediction — related
- Native speakers rated outputs of LLMs fine-tuned on Nigerian Pidgin English as understandable and context-aware, though no non-fine-tuned baseline was evaluated — related
- A fine-tuned Qwen3.5-9B simulated-respondent model predicts item difficulty with Pearson correlation 0.85, exceeding MathBERT (0.68) and MetaMath (0.75) baselines — related
- The EduQwen 32B models achieve up to 96.52% accuracy on the CDPK Benchmark, surpassing the previous leaderboard leader Gemini-3 Pro at 90.55% — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
model-size-and-llm-vs-mlm-performance-vocabulary-difficulty |
title |
In the fine-tuning experiments, increasing base model size up to 32B parameters improved performance, and larger LLMs consistently outperformed smaller MLMs |
description |
In the fine-tuning experiments, increasing base model size up to 32B parameters improved performance, and larger LLMs consistently outperformed smaller MLMs |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2026 |
recheck_by |
2029 |
generated |
by process:wiki-ingest on 2026-10-11 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — adam-nohejl-2026 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.