LLM coders replicate many human discursive moves but lack questioning, adaptive responsiveness, and emotional expressiveness¶

Claim · All claims
Evidence · 1 study (2 entries) · qualitative r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? LLMs emulate basic human interaction patterns including stance maintenance, acknowledgment, evidence reassessment, contrast, tentative stances, and alternative proposals. → Jeongyeon Kim and John Mitchell 2026 q2 i? LLMs engage in significantly less questioning than humans and lack adaptive responsiveness to evolving conversational context. → Jeongyeon Kim and John Mitchell 2026 (2)

Evidence¶

Jeongyeon Kim and John Mitchell 2026¶

Jeongyeon Kim and John Mitchell. (2026). How AI Coders Discuss, Disagree, and Reach Consensus: Challenges and Opportunities for LLM-Based Qualitative Coding. arXiv. https://arxiv.org/abs/2609.11109 doi:10.48550/arxiv.2609.11109

q2 · i? · qualitative · r2

Qualitative coding of AI-agent discussion transcripts using the Thomas-Kilmann Conflict Mode Instrument and discursive-move frameworks, compared with human moves from the literature (Table 6). The authors report LLMs also support "collaborative reflection and seeking consensus."

"LLMs emulate basic patterns of interaction, such as maintaining stances, acknowledging others’ perspectives, reassessing evidence, explicitly contrasting ideas, expressing a tentative stance, and proposing alternatives."

Jeongyeon Kim and John Mitchell 2026 (2)¶

Jeongyeon Kim and John Mitchell. (2026). How AI Coders Discuss, Disagree, and Reach Consensus: Challenges and Opportunities for LLM-Based Qualitative Coding. arXiv. https://arxiv.org/abs/2609.11109

q2 · i? · qualitative · r2

Qualitative comparison of AI-agent and human discursive moves. The authors find LLMs "engage in significantly less questioning compared to humans" and operate within a more fixed framework, missing adaptability to implicit cues and evolving conversational dynamics.

"It is worth noting that LLMs engage in significantly less questioning compared to humans. While humans frequently ask clarifying or probing questions to refine understanding or actively explore the perspectives of other participants, LLMs tend to prioritize delivering coherent and direct answers over engaging in exploratory dialogue and mutual learning."

Discussion¶

Related Claims¶

• Human-LLM sentiment divergences were patterned: the LLM tended to assign affective labels to statements a human coder read as neutral — related
• LLM-based transcript classifications agreed highly with human judgment for explicit questions (κ = .778) and math talk (κ = .722), but only moderately for need for help (κ = .562) and confusion (κ = .531) — related
• Standalone LLM tutors replicate Phase II's profile: strong single-interaction intelligence without systematic control over the learning trajectory
• The adaptive virtual patient produces a steadily climbing disclosure trajectory across a session while a prompt-only baseline with the same LLM and persona stays flat
• Ignoring the emotion strategy neither recognizes nor explores the displayed emotion, closing the discursive space
• In blind A/B comparison, raters prefer personalised (Adaptive) questions over non-personalised (Random-topic) questions about 68–69% of the time in Biology and Chemistry
• ChatGPT served as an emotional support agent during practicum, offering a nonjudgmental, constantly available interlocutor — related
• In blind verification, model selection matters more than source type: Bradley-Terry ranking interleaves human coders and LLMs, with per-model win rates spanning significant human advantage to numerical LLM advantage — related
• LLMs are more intrusive and more consistent than humans, eager to view neutral discussions as opportunities for positive reinforcement — related
• LLM participant agents clarify stage goals, keep discussions on topic, and prompt deeper ethical reflection — related
• Students engaged deeply with LLM feedback, with coded evidence of reflection, rebuttal, and acceptance in the large majority of responses — related