Source provenance labels functioned as a seal of authority: 11 of 12 participants failed to detect a planted source-card mismatch¶
Claim · All claims
Evidence · 1 study (2 entries) · causalr1–r2·q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? In a controlled deception experiment, only 1 of 12 participants detected the source-card mismatch; the remaining 11 accepted it without hesitation. → Ruqi Sun 2026
q2 i? Perceived trust trended higher under VeriForge (M=5.92 vs M=4.75) but did not reach significance (p=.125), while perceived authorial autonomy was identical (M=6.17 vs M=6.17; p=1.0). → Ruqi Sun 2026 (2)
Evidence¶
Ruqi Sun 2026¶
Ruqi Sun, Jiaping Li, Wenhui Tao, Ximing Zheng, Yuefeng Tan, Jiahao Wei, and Yuxin Ma. (2026). VeriForge: Mitigating Latent Knowledge Gaps in Narrative Drafting via Mixed-Initiative Scaffolding. https://arxiv.org/abs/2608.09698 doi:10.48550/arxiv.2608.09698
q2 · i? · causal · r2
Controlled deception in the within-subjects study: the source passage on the second Knowledge Card was replaced with an unrelated passage while keeping the AI-generated description unchanged, creating a detectable mismatch. The test measured evidence inspection, not hallucination detection.
"Only 1 of 12 participants detected the source-card mismatch. The remaining 11 did not notice."
Ruqi Sun 2026 (2)¶
Ruqi Sun, Jiaping Li, Wenhui Tao, Ximing Zheng, Yuefeng Tan, Jiahao Wei, and Yuxin Ma. (2026). VeriForge: Mitigating Latent Knowledge Gaps in Narrative Drafting via Mixed-Initiative Scaffolding. https://arxiv.org/abs/2608.09698
q2 · i? · causal · r1
Exploratory RQ3 survey measures excluded from FDR correction. The authors attribute the compressed trust difference to both conditions sharing the same source-attributed retrieval backend.
"Perceived trust trended higher under VeriForge (𝑀= 5.92vs. 𝑀= 4.75) but did not reach significance (𝑝=. 125)"
Discussion¶
Related Claims¶
- Surface validation can function while substantive validation fails: most participants detected flawed AI recommendations yet continued following them — related
- Experimental-group students detected AI errors and enacted verification and ethical self-regulation as routine practices — related
- Student experiences with GenAI in design thinking fall into four themes: perceived benefits, ethical concerns, hesitance and acceptance, and critical validation — related
- Perfect AI execution cannot remove four limits: accuracy does not settle end-setting, safety does not confer legitimacy, detection does not secure integrity, and outputs do not replace participation — a broader claim this one bears on
- VeriForge raises perceived creativity support and lowers workload relative to a same-retrieval baseline, with mixed survival after correction — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
provenance-paradox-uncritical-trust |
title |
Source provenance labels functioned as a seal of authority: 11 of 12 participants failed to detect a planted source-card mismatch |
description |
Source provenance labels functioned as a seal of authority: 11 of 12 participants failed to detect a planted source-card mismatch |
status |
review |
evidence_strength |
weak |
newest_evidence |
2026 |
generated |
by process:wiki-ingest on 2026-10-11 |
trust |
unverified — no one has confirmed this page's content |
sources |
2 — ruqi-sun-2026, ruqi-sun-2026-2 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.