Data scarcity and long-form generation remain bottlenecks: on Prompt 8 prompting outperforms SFT and GRPO¶

Claim · All claims
Evidence · 1 study · causal r2 · q2
Currency · newest study 2026 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? On Prompt 8, which has only 723 training essays and requires substantially longer responses (average length ≈650), prompting-based methods outperform both SFT and GRPO. → Heejin Do 2026

Evidence¶

Heejin Do 2026¶

Heejin Do, Jakub Kontak, Mrinmaya Sachan. (2026). SWIM: Student Writing Simulation via Proficiency-Conditioned Generation. arXiv preprint arXiv:2609.03215. https://arxiv.org/abs/2609.03215 doi:10.48550/arxiv.2609.03215

q2 · i? · causal · r2

Discussion-section analysis of the clearest failure case, Prompt 8 in Table 5, where prompting-based methods outperform both SFT and GRPO. The paper reports Prompt 8 "contains only 723 training essays (less than half the size of most other prompts)" with average length ≈650.

"Prompt 8 contains only 723 training essays (less than half the size of most other prompts) while requiring substantially longer responses (average length ≈650 )."

Discussion¶

Related Claims¶

• Zero-shot prompting collapses proficiency variation toward an idealized high-scoring student population, while SFT and GRPO match the human score and length distributions — related
• GRPO with the Proficiency Alignment Reward improves alignment over SFT across every writing trait and essay prompt — reports the opposite
• Supervised fine-tuning on real score–essay pairs substantially outperforms prompting for proficiency-conditioned writing simulation — reports the opposite
• SFT and GRPO alignment gains reflect learned proficiency-conditioned generation rather than memorization or output collapse — related
• GRPO-with-PAR gains over SFT transfer to independent evaluators never used during policy optimization — reports the opposite
• Models reproduce high-proficiency writing more faithfully than authentic low-proficiency linguistic form, with distinct failure modes by method — related
• Rubric-grounded prompting alone provides limited proficiency control, aligning far better on content-oriented traits than on form-oriented traits — related