The AVP's generated text matches its intended internal disclosure level about twice as often as the static baseline (0.651 vs. 0.311 agreement)¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? Agreement between the system's intended disclosure level and an independent evaluator's judgment of the generated utterance was 0.651 for the AVP versus 0.311 for the static VP. → Angela Chen 2026

Evidence¶

Angela Chen 2026¶

Angela Chen, Siwei Jin, Catherine Bao, Canwen Wang, Robert E. Kraut, Tongshuang Wu, and Haiyi Zhu. (2026). The Empirically Grounded Adaptive Virtual Patient for Psychotherapy Training: Disclosure That Responds to Therapist Micro-Skills. https://arxiv.org/abs/2606.10051 doi:10.48550/arxiv.2606.10051

q2 · i? · design · r2

Evaluation of state-output alignment using an independent LLM disclosure-labeler as ground truth across the 80 recorded sessions; "Adaptive overall agreement is 0.651 vs. Static’s 0.311". Per-level precision for AVP H-level was low (.326), printed in Table 7.

"Figure 3 shows the confusion matrices behind the alignment metric: Adaptive overall agreement is 0.651 vs. Static’s 0.311."

Discussion¶

Related Claims¶

• The adaptive virtual patient produces a steadily climbing disclosure trajectory across a session while a prompt-only baseline with the same LLM and persona stays flat — related
• In the AVP, per-turn disclosure change responds to trainee exploration but not measurably to trainee empathy, and the static baseline shows no significant joint response — related
• The deployed empirically motivated weight vector outperforms equal, empathy-only, and exploration-only weightings in evaluator agreement, with exploration carrying most of the adaptive signal — related