Participants fail to distinguish undisclosed AI teammates from human teammates above chance in synchronous text-based group interaction¶

Claim · All claims
Evidence · 1 study · causal r2 · q3
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q3 i? Across 1,572 identity judgments by 786 participants, signal detection sensitivity for distinguishing AI from human teammates was near chance (d' = 0.117). → Lixiang Yan 2026

Evidence¶

Lixiang Yan 2026¶

Lixiang Yan, Yueqiao Jin, Xibin Han, Dragan Gasevic. (2026). Socially fluent AI decouples conversational signals from source identity in online interaction. https://arxiv.org/abs/2605.23426 doi:10.48550/arxiv.2605.23426

q3 · i? · causal · r2

Signal detection analysis of 1,572 post-interaction identity judgments from 786 participants in 471 groups showed "identity discrimination sensitivity collapses to chance levels" (d' = 0.117, 95% CI [−0.007, 0.257]); hit rate 0.317 closely matched false alarm rate 0.276.

"Across 1,572 identity judgments (685 AI targets; 887 human targets) provided by 786 participants nested within 471 collaborative groups, identity discrimination sensitivity collapses to chance levels. Signal detection analysis yielded a sensitivity of 𝑑′ = 0.117, 95% CI [−0.007, 0.257]"

Discussion¶

Related Claims¶

• Chance-level identity discrimination is uniform across participants and not associated with AI literacy, personality, or demographics — a narrower finding that bears on this claim
• Judgment-truth dissociation: the same cues that predict ground-truth identity are only weakly incorporated into explicit identity judgments — a narrower finding that bears on this claim
• Dialogue-derived interactional cues robustly encode ground-truth identity, achieving near-ceiling AI identification under cross-validation — reports the opposite
• In a human Turing test, participants could not reliably distinguish BEAGLE traces from real student data (52.8% accuracy; d′ = 0.15 within the ±0.3 equivalence bound, pTOST = 0.038) — related
• Representational similarity analysis shows identity judgments are organised around subjective impressions, orthogonal to diagnostic cue structure — a narrower finding that bears on this claim
• Participants remain socially sensitive to conversational stance even while categorical identity inference fails — related
• Admissions officers can often discriminate AI-written from human-written essays, achieving an AUC of 0.70, well above chance but below commercial detectors — related