SWIM: proficiency-conditioned essay generation and proficiency-alignment evaluation¶
Research Method · All research methods
Evidence · 5 claims (4 for, 1 mixed) · 1 study (1 causal),q2· 0 of 1 report an effect size · 5 claims rest on one study
Description¶
SWIM formalizes student writing simulation as proficiency-conditioned essay generation: given a writing prompt and a target trait-score profile, a generator produces an essay exhibiting the specified writing behavior, and "an AES-based evaluation framework for" measuring profile alignment scores the output with a frozen multi-trait AES verifier using QWK. The authors train generators via SFT on real (essay, profile) pairs and then refine them with GRPO using the Proficiency Alignment Reward, a dense per-sample reward based on trait-normalized accuracy against the target profile.
Accounts¶
- SWIM: proficiency-conditioned essay generation task with AES-based profile-alignment evaluation and the Proficiency Alignment Reward: SWIM formalizes student writing simulation as proficiency-conditioned essay generation: given a writing prompt and a target trait-score profile, a generator produces an essay exhibiting the specified writing behavior, and "an AES-based evaluation framework for" measuring profile alignment scores the output with a frozen multi-trait AES verifier using QWK. The authors train generators via SFT on real (essay, profile) pairs and then refine them with GRPO using the Proficiency Alignment Reward, a dense per-sample reward based on trait-normalized accuracy against the target profile. (Heejin Do et al. (2026))
Claims¶
- Supervised fine-tuning on real score–essay pairs substantially outperforms prompting for proficiency-conditioned writing simulation [+M]
- GRPO with the Proficiency Alignment Reward improves alignment over SFT across every writing trait and essay prompt [+M]
- SFT and GRPO alignment gains reflect learned proficiency-conditioned generation rather than memorization or output collapse [+W]
- Data scarcity and long-form generation remain bottlenecks: on Prompt 8 prompting outperforms SFT and GRPO [~W]
- Zero-shot prompting collapses proficiency variation toward an idealized high-scoring student population, while SFT and GRPO match the human score and length distributions [+W]
Related Research Methods¶
-
Key Sources¶
- Heejin Do, Jakub Kontak, Mrinmaya Sachan. (2026). SWIM: Student Writing Simulation via Proficiency-Conditioned Generation. arXiv preprint arXiv:2609.03215. https://arxiv.org/abs/2609.03215
Page metadata
| Field | Value |
|---|---|
type |
research-method |
id |
swim-proficiency-conditioned-essay-generation-and-proficiency-alignment-evaluation |
title |
SWIM: proficiency-conditioned essay generation and proficiency-alignment evaluation |
description |
SWIM formalizes student writing simulation as proficiency-conditioned essay generation: given a writing prompt and a target trait-score profile, a generator produces an essay exhibiting the specified writing behavior, and "an AES-based evaluation framework for" measuring profile alignment scores the |
status |
review |
generated |
by process:settle-candidates on 2026-10-11 |
trust |
unverified — no one has confirmed this page's content |
Frontmatter is the page's machine-readable record — see the Schema & Guide.