Users' baseline ranking accuracy significantly predicts the quality of the AI assistant's advice¶
Claim · All claims
Evidence · 1 study · causalr2·q3
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q3 i? Baseline ranking accuracy significantly predicted AI advice quality (b = 0.478, p = .008), indicating lower-quality initial responses lead to poorer AI advice. → Cansu Koyuturk 2026
Evidence¶
Cansu Koyuturk 2026¶
Cansu Koyuturk, Sabrina Guidotti, and Dimitri Ognibene. (2026). The Hidden Cost of Contextual Sycophancy: an AI Literacy Intervention in Human–AI Collaboration. https://arxiv.org/abs/2605.18372 doi:10.48550/arxiv.2605.18372
q3 · i? · causal · r2
Regression of AI advice quality (NDCG@6 of the assistant's top-6 recommendations, extracted via an LLM-as-judge pipeline) on baseline ranking accuracy and condition in the 60-participant experiment. The article reports "b = 0.478, SE = 0.180, z = 2.65, p = .008, 95% CI [0.125, 0.831]".
"Baseline accuracy significantly predicted advice quality, b = 0.478, SE = 0.180, z = 2.65, p = .008, 95% CI [0.125, 0.831]."
Discussion¶
Related Claims¶
- Carryover of user errors into AI advice reduces both advice quality and final ranking accuracy — related
- Users' baseline ranking accuracy significantly predicts their final ranking accuracy after AI collaboration, with no other significant effects — related
- The sycophancy-focused intervention significantly reduced the assistant's positional mirroring of incorrect user rankings and rank-order alignment — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
baseline-accuracy-predicts-advice-quality |
title |
Users' baseline ranking accuracy significantly predicts the quality of the AI assistant's advice |
description |
Users' baseline ranking accuracy significantly predicts the quality of the AI assistant's advice |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2026 |
recheck_by |
2029 |
generated |
by process:wiki-ingest on 2026-10-10 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — cansu-koyuturk-2026 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.