Contrary to expectations, agent-led reflection sessions typically ended with participants sounding more certain of their plans than when they started¶

Claim · All claims
Evidence · 1 study · associational r2 · q2
Currency · no dated study recorded · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Within Study 2 sessions, participant certainty dipped mid-session and rose toward the end: first turns averaged 4.11 and last turns 4.32 on the five-point scale, with 36% of sessions ending higher and 24% lower. → certainty-arc

Evidence¶

certainty-arc¶

Subigya K. Nepal, Serena Soh, Noah Vinoya, SoHyun Park, Mahnaz Roshanaei, and Gabriella Harari. 2027. Faithful Where It Can Be Checked: Auditing a Reflection Agent Against Its System Prompt in a Randomized Trial. In Proceedings of CHI Conference on Human Factors in Computing Systems (CHI '27). ACM, New York, NY, USA. https://arxiv.org/abs/2609.19635 doi:10.48550/arxiv.2609.19635

q2 · i? · associational · r2

Turn-level certainty ratings (validated against human coders, MAE .32) across Study 2 sessions, shown in Figure 3. The authors expected conversations to open decisions up without closing them; the measure showed the opposite arc.

"Within a session, participants' certainty dipped a little in the middle and rose toward the end, and sessions typically ended with participants sounding more sure than when they started (first turns averaged 4.11 on the five-point scale, last turns 4.32; 36% of sessions ended higher and 24% ended lower)."

Discussion¶

Related Claims¶

• The agent behavior linked to the trial's worse career outcomes was repeated demands to decide: participants pressed most ended most doubtful — related
• Decisions participants volunteered were worded just as firmly as decisions the agent demanded, and revisions of earlier positions were rare — related
• When participants declined to decide and the agent asked again, the resulting commitments were the weakest in the corpus — related
• Public regard decreased linearly over time while reported discrimination dipped at Time 2 and rose again at Time 3 — related
• A deployed reflection agent followed its prompt's countable rules but broke its dispositional rules about manner — related