Fine-tuning with INSIDE internal dialogue achieves the lowest Wasserstein distances to real student code distributions on test_OP across all action-fidelity metrics¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? On test_OP, models fine-tuned with INSIDE produce code distributions closest to real student submissions across pass rate, LOC, AST depth/width, and PEP 8 metrics, outperforming both SFT and prompting baselines. → Rose Niousha 2026

Evidence¶

Rose Niousha 2026¶

Rose Niousha, Minwoo Kang, Narges Norouzi. (2026). INSIDE the Student's Mind: Jointly Modeling Latent Reasoning and Action in LLM Student Simulators. Published as a conference paper at COLM 2026. https://arxiv.org/abs/2608.10492 doi:10.48550/arxiv.2608.10492

q2 · i? · design · r2

Distributional evaluation comparing fine-tuned and prompted LLMs against real student submissions on test_OP (5,262 submissions from Spring 2024 students), reporting Wasserstein distance over pass rate, LOC, AST, and PEP 8 metrics. The article reports INSIDE "consistently achieves the lowest Wasserstein distances across all metrics"; e.g., pass-rate distance 0.05 for INSIDE models vs 0.14-0.15 for SFT.

"Ontest_OP (Table 1a), INSIDE consistently achieves the lowest Wasserstein distances across all metrics, indicating the closest match to real student code distributions."

Discussion¶

Related Claims¶

• INSIDE fine-tuning achieves the highest alignment between generated internal dialogue and ground-truth student code edits, up to 57.9% — related
• Prompting-based student simulators maintain artificially high pass rates (~80%) from the first submission step, while fine-tuned models track real students' low-start, sharp-increase trajectories — related
• High self-consistency between a simulator's reasoning and its own generated code does not imply alignment with real student reasoning or behavior — related
• Differences in student pass-rate distributions between test splits explain the smaller INSIDE-over-SFT improvement on test_NP — related
• GPT-generated feedback passes simulated-student helpfulness evaluations only about 35% of the time on real student data — related
• LLM-generated erroneous code aligns with authentic student errors at levels that depend on model and prompting strategy, with Claude Sonnet 4 most consistently aligned — related