Impression narratives rely on weakly diagnostic suspicion heuristics such as response speed, fluency, and perceived scriptedness¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r2 · q3
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q3 i? BERTopic analysis of 1,568 impression texts found a dominant generic humanness topic (59.6% of texts) and low mutual information between topic membership and identity judgment (MI = 0.089). → Lixiang Yan 2026 q3 i? TF-IDF contrasts show terms in incorrect judgments emphasised latency and surface fluency, while odds ratios contrasting incorrect versus correct judgments were close to unity for most topics. → Lixiang Yan 2026 (2)

Evidence¶

Lixiang Yan 2026¶

Lixiang Yan, Yueqiao Jin, Xibin Han, Dragan Gasevic. (2026). Socially fluent AI decouples conversational signals from source identity in online interaction. https://arxiv.org/abs/2605.23426 doi:10.48550/arxiv.2605.23426

q3 · i? · causal · r2

Unsupervised BERTopic modelling of 1,568 impression texts extracted eight topics; "a single dominant topic capturing generic humanness and fluency language accounted for 59.6% of texts". Mutual information between topic membership and judgment was low (MI = 0.089).

"Topic prevalence was strongly skewed: a single dominant topic capturing generic humanness and fluency language accounted for 59.6% of texts (Topic 0, 95% CI [57.1%, 62.0%]), with the remaining topics forming a long tail of smaller, more specific themes"

Lixiang Yan 2026 (2)¶

Lixiang Yan, Yueqiao Jin, Xibin Han, Dragan Gasevic. (2026). Socially fluent AI decouples conversational signals from source identity in online interaction. https://arxiv.org/abs/2605.23426

q3 · i? · causal · r2

Supervised class-based TF-IDF analysis showed incorrect or AI-attributing judgments emphasised "response latency and surface fluency" and "repetition or scriptedness", while correct Human judgments referenced socially interpretable qualities; odds ratios were close to unity for most topics (e.g., Topic 0: OR = 0.853).

"Distinctive terms in incorrect or AI-attributing categories emphasised response latency and surface fluency (e.g., “fast”, “quick”), repetition or scriptedness (e.g., “repeating”, “script”), and disagreement markers (e.g., “disagree”)."

Discussion¶

Related Claims¶

• Judgment-truth dissociation: the same cues that predict ground-truth identity are only weakly incorporated into explicit identity judgments
• Representational similarity analysis shows identity judgments are organised around subjective impressions, orthogonal to diagnostic cue structure
• Topic modelling yielded 16 topics dominated by four: AI in Education & Pedagogy (39.43%), Applied ML & Data Science (21.33%), AI Ethics Risk & Governance (20.83%), and ChatGPT for Teaching, Writing & Coding (7.13%) — related
• Students tend to rely on non-diagnostic cues when judging their own performance, resulting in inaccurate judgments — related
• Teachers often rely on informal impressions rather than systematic, theory-grounded diagnostic procedures when judging students' SRL — related