Humans are more certain when judging that facilitation is not needed than when judging it is¶

Claim · All claims
Evidence · 1 study · associational r1 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? For no-reinforcement judgments, humans are more certain than LLMs, and both humans and LLMs show high uncertainty for negative reinforcement. → Dimitris Tsirmpas 2026

Evidence¶

Dimitris Tsirmpas 2026¶

Dimitris Tsirmpas, Katerina Korre, John Pavlopoulos. (2026). To Facilitate or not to Facilitate: Human and LLM Facilitator Tendencies in Online Discussions. https://arxiv.org/abs/2607.28643 doi:10.48550/arxiv.2607.28643

q2 · i? · associational · r1

Analysis of certainty distributions per facilitation label (Fig. 3) from the same expert survey and LLM survey. The authors report that "for no reinforcement there is a clear difference with humans being more certain that LLMs."

"For negative reinforcement, both humans and LLMs show high uncertainty, while for no reinforcement there is a clear difference with humans being more certain that LLMs."

Discussion¶

Related Claims¶

• Gemini-2.5-Pro can effectively replicate human expert judgments when answering SLM-generated MCQs — related
• Blind expert verification shows no overall preference for human over LLM qualitative coding when sources are judged symmetrically — related
• LLM-as-judge automated evaluation can achieve human-level agreement when carefully validated — a broader claim this one bears on
• Human-LLM agreement remains low across constructs (κ = 0.000 to 0.419), with self-efficacy showing the best alignment and Prior KSAs none — related
• Using AI to evaluate AI outputs was often challenging because AI act as yes people and humans struggled to agree on evaluation rubrics — related