Engage-to-Unlock improved error-classification accuracy over Standard Chatbot but showed no significant advantage in overall judgment accuracy or complete diagnosis¶

Claim · All claims
Evidence · 1 study · causal r2 · q3
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? Among detected flaws, Engage-to-Unlock participants classified errors more accurately than Standard Chatbot participants (90.43% versus 78.12%, pHolm=.003), but not significantly more than Time-Matched Unlock participants (83.58%, pHolm=.051). → Xiaotian Su 2026

Evidence¶

Xiaotian Su 2026¶

Xiaotian Su, Laura Rimell, Jiazheng Li, Amal Rannen-Triki, Ulrich Paquet, Lisa Anne Hendricks, Rida Qadri, Daphne Ippolito and Piotr Mirowski. (2026). Who Thinks First? Designing Productive Friction with Engage-to-Unlock GenAI. https://arxiv.org/abs/2610.01518 doi:10.48550/arxiv.2610.01518

q3 · i? · causal · r2

Binomial GLMM with crossed random intercepts for participant and passage on the 779 detected-flaw trials from the N=398 experiment. Overall judgment accuracy ranged from 61.61% to 66.36% with no significant differences between conditions, and neither complete-diagnosis contrast was significant.

"Participants in Engage-to-Unlock classified detected errors more accurately than those in Standard Chatbot (90.43% versus 78.12%). The planned GLMM contrast indicated higher odds of correct classification under Engage-to-Unlock (𝑝Holm =. 003)."

Discussion¶

Related Claims¶

• Engage-to-Unlock participants completed passage evaluation faster and with higher accuracy-per-time efficiency than both AI-access comparison conditions — related
• Engage-to-Unlock redistributed effort across tasks: participants spent more time writing and less time evaluating, without increasing overall task duration — related