Engage-to-Unlock participants completed passage evaluation faster and with higher accuracy-per-time efficiency than both AI-access comparison conditions¶

Claim · All claims
Evidence · 1 study · causal r2 · q3 · i2 medium
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i2 Engage-to-Unlock participants completed evaluation in 17.41 minutes on average versus 22.21 (Standard Chatbot, d=−0.48) and 20.29 (Time-Matched Unlock, d=−0.30), and showed higher overall judgment efficiency than Standard Chatbot (d=0.53). → Xiaotian Su 2026

Evidence¶

Xiaotian Su 2026¶

Xiaotian Su, Laura Rimell, Jiazheng Li, Amal Rannen-Triki, Ulrich Paquet, Lisa Anne Hendricks, Rida Qadri, Daphne Ippolito and Piotr Mirowski. (2026). Who Thinks First? Designing Productive Friction with Engage-to-Unlock GenAI. https://arxiv.org/abs/2610.01518 doi:10.48550/arxiv.2610.01518

q3 · i2 · causal · r2

Planned contrasts on Task 2 accuracy-per-minute measures in the N=398 between-participants experiment; overall judgment efficiency d=0.53, conditional error-classification efficiency d=0.60, complete diagnosis efficiency d=0.48. Relative to Time-Matched Unlock, "the contrast was significant for conditional error-classification efficiency (𝑝Holm =. 015, 𝑑= 0.38), but not for overall judgment efficiency (𝑝Holm =.290)".

"Compared with Standard Chatbot, Engage-to-Unlockshowedhighervaluesacrossallthreemeasures: overalljudgmentefficiency (𝑝Holm =. 0003,𝑑= 0.53), conditional error-classification efficiency (𝑝Holm =. 0003,𝑑= 0.60), and complete diagnosis efficiency (𝑝Holm =. 0010, 𝑑= 0.48)."

Discussion¶

Related Claims¶

• Engage-to-Unlock improved error-classification accuracy over Standard Chatbot but showed no significant advantage in overall judgment accuracy or complete diagnosis — related
• Time-matched unlocking did not reproduce the engagement state that triggered unlocks: most Time-Matched Unlock participants received access before meeting the discourse criteria — related
• Engage-to-Unlock redistributed effort across tasks: participants spent more time writing and less time evaluating, without increasing overall task duration — related
• AI-tutored students spend less time on task than the 60-minute in-class lesson, with no correlation between time and scores — related