An explicit prompt instruction to challenge gently changed nothing: the agent challenged in about 1% of turns under both prompts¶

Claim · All claims
Evidence · 1 study · causal r1 · q2
Currency · no dated study recorded · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Challenge occurred in about 1% of agent turns in both studies, although only Study 2's prompt asked for gently challenging follow-ups; over half of Study 2 participants received no challenging turn at all. → challenge-rates

Evidence¶

challenge-rates¶

Subigya K. Nepal, Serena Soh, Noah Vinoya, SoHyun Park, Mahnaz Roshanaei, and Gabriella Harari. 2027. Faithful Where It Can Be Checked: Auditing a Reflection Agent Against Its System Prompt in a Randomized Trial. In Proceedings of CHI Conference on Human Factors in Computing Systems (CHI '27). ACM, New York, NY, USA. https://arxiv.org/abs/2609.19635 doi:10.48550/arxiv.2609.19635

q2 · i? · causal · r1

Cross-study comparison of the same GPT-4o model under two prompts (facilitator vs. conversation partner), from the fidelity results. Challenge was 1.1% in Study 1 and 1.2% in Study 2 (Table 6), and 55% of participants never received any challenge.

"And challenge sat at about 1% in both studies, even though only Study 2 asked for it. That last comparison is the cleanest evidence in the paper: an entire instruction, present in one prompt and absent from the other, changed nothing."

Discussion¶

Related Claims¶

• A deployed reflection agent followed its prompt's countable rules but broke its dispositional rules about manner — related
• The agent's praise was indiscriminate affirmation: validation followed about half of participant turns regardless of what the participant said — related