Few-shot prompting does not uniformly improve LLM performance and can degrade subskill mastery estimates, most notably for phonology¶

Claim · All claims
Evidence · 1 study · causal r1 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? Few-shot prompting can slightly degrade subskill mastery, particularly for phonology, in enhanced models such as Qwen2.5-VL CrossAttn. → Shangchao Min 2026

Evidence¶

Shangchao Min 2026¶

Shangchao Min, Shaonan Wang, Xinyu Gao, Hui Wang, Zhiling Jin, and 2 more. (2026). Standardized Assessment of LLM English Proficiency. https://doi.org/10.21203/rs.3.rs-8820245/v1

q2 · i? · causal · r1

Authors' discussion of prompting-condition comparisons across the model suite on CSEBench. The article states few-shot prompting "slightly degrades subskill mastery, most notably for phonology", visible in enhanced models such as Qwen2.5-VL CrossAttn. No effect size is printed.

"Few-shot prompting does not uniformly improve performance and, in some cases, slightly degrades subskill mastery, most notably for phonology."

Discussion¶

Related Claims¶

• Closed-source LLMs reach CSE Level 6 while most open-source baselines cluster at CSE Levels 3–4 on CSEBench — related
• Targeted enhancements (retrieval-augmented inference, multimodal training) substantially raise open-source models' phonology mastery to near-ceiling levels — related
• Open-source LLMs show persistent phonology deficits despite adequate mastery of other subskills — related
• In-context learning did not consistently improve LLM annotation reliability over zero-shot prompting for engagement coding — a broader claim this one bears on
• Few-shot prompting modestly improves LLM-human score agreement, inconsistently across datasets and models — related