Multi-turn RL training in simulated clinical encounters improves diagnostic accuracy under adversarial conditions, raising rubric-based accuracy from 81.0% to 88.0%¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2026 · checked against its sources 2026-10-18 (automated, 1 of 1 entry) · not reviewed by a person

Subclaims¶

q2 i? On held-out in-domain adversarial scenarios, the ResidencyRL-trained agent achieves higher diagnostic accuracy than the base model (88.0% vs. 81.0% rubric score ≥4/5). → Liévin 2026

Evidence¶

Liévin 2026¶

Liévin, V., Schmidgall, S., Strother, T., Bijamov, A., Goel, A., Palepu, A., Park, C., Balazadeh, V., Sun, M. W., Guerard, M., Chen, J., Steiner, D., Dhillon, V., Azar, I., Mehta, A., Spetsieris, N., Shah, S., Abdelrahim, M., Dahiya, A., Liu, Y., Chou, K., Matias, Y., Hassidim, A., Webster, D. R., Le, Q. V., Hadsell, R., Barral, J., Radebaugh, C., Faust, A., Azizi, S., Schaekermann, M., Chen, P.-H. C., Tu, T., Racz, D., & Yang, L. (2026). ResidencyRL: Reinforcement Learning in Simulated Clinical Environments. arXiv. https://arxiv.org/abs/2608.07418 doi:10.48550/arxiv.2608.07418

q2 · i? · design · r2

Held-out evaluation on 200 adversarial safety scenarios generated by the same pipeline as training but excluded from it via TF-IDF filtering. The article reports the agent "improves over the base model across all targeted metrics, including diagnostic accuracy" with rubric score ≥4/5 rising from 81.0% to 88.0%. No effect size is printed.

"On held-out in-domain cases, the agent improves over the base model across all targeted metrics, including diagnostic accuracy (rubric score≥4/5: 81.0% to 88.0% under adversarial conditions)"

Discussion¶

Related Claims¶

• Procedural competencies from simulation-based RL training generalize to out-of-domain benchmarks including AMIE Mx, AgentClinic, and CRAFT-MD — related
• RL training reduces missed red flag rates by approximately one third under adversarial conditions, mitigating premature closure — related
• Blinded board-certified clinicians prefer the RL-trained agent in 87.6% of side-by-side comparisons for overall clinical impression — related