Two-phase cross-prompt pre-finetuning with key phrases significantly improves short answer scoring accuracy when in-prompt training data is limited¶

Claim · All claims
Evidence · 1 study · causal r2 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Pre-finetuning with key phrases yields a maximum improvement of approximately 0.25 QWK when only 10 answers are used for finetuning, and halves the training data needed for the same performance. → Funayama 2025

Evidence¶

Funayama 2025¶

Funayama, H., Matsubayashi, Y., Asazuma, Y., Mizumoto, T., & Inui, K. (2025). Cross-prompt Pre-finetuning of Language Models for Short Answer Scoring. Int J Artif Intell Educ. https://doi.org/10.1007/s40593-025-00474-w

q2 · i? · causal · r2

Experiments on the RIKEN Japanese SAS dataset compared five BERT-based settings (baseline, key phrases only, 2-phase with/without key phrases, ensemble) across 10-200 finetuning instances. The article reports a maximum improvement of "approximately 0.25 QWK" at 10 answers, and that pre-finetuning with key phrases "reduced the required training data by half while maintaining the same performance."

"The model performance was significantly improved when the training data were limited, with a maximum improvement of approximately 0.25 QWK when 10 answers were used for finetuning compared with the performance of the baseline."

Discussion¶

Related Claims¶

• Cross-prompt finetuning of a pre-finetuned 8B LLM on 50 instances per prompt achieves average QWK of 0.85, comparable to professional human graders (0.87) — related
• Pre-finetuning without key phrases does not improve scoring performance and slightly lowers it relative to the BERT baseline — related
• Cross-prompt pre-finetuning shows no significant benefit when sufficient (200-answer) in-prompt training data is available — related
• LLMs cannot effectively perform in-context learning for short answer scoring, but cross-prompt pre-finetuning substantially boosts their zero-shot performance — related
• Pre-finetuning performance increases with the number of distinct prompts used, holding total pre-finetuning answers fixed — related
• The pre-finetuned model learns a scoring principle: predicted scores correlate strongly and negatively with edit distance between answers and key phrases — related