Token consumption is uncorrelated with task quality across models (r = −0.03), suggesting agents lack mechanisms for knowing when to stop¶

Claim · All claims
Evidence · 1 study · associational r? · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? Pooling all 480 model–task evaluations, the Pearson correlation between token consumption and task score is r = −0.03 (p = 0.49), effectively zero; even within individual models no correlation exceeds |r| = 0.08. → Junjie Yu 2026

Evidence¶

Junjie Yu 2026¶

Junjie Yu, Pengrui Lu, Weiye Si, Hongliang Lu, Jiabao Wu, Kaiwen Tao, Kun Wang, Lingyu Yang, Qiran Zhang, Xiuting Guo, Xuanyu Wang, Yang Wang, Yanjie Wang, Yi Yang, Zijian Hu, Ziyi Yang, Zonghan Zhou, Binghao Qiang, Borui Zhang, Chenning Li, Enchang Zhang, Feifan Chen, Feng Jian, Fengyin Sun, Hao Qiu, Hao Zheng, Haoran Zhu, Hongyu Liu, Jianbin Deng, Jiaxin Song, Jiaying Chi, Jiayou Shi, Jie Fang, Jinghui Zhong, Jingyu Zhou, Jinze Li, Junfeng Yi, Junyan Yu, Junzhi Xue, Ni Song, Pengyi Chen, Qi Chen, Quansheng Li, Rui Tao, Shenghai Gong, Shenhang Lu, Tianqi Shen, Tianxiang Zhu, Tiehan Kang, Tingyu Li, Wendi Wu, Xiao Shen, Xiao Zhou, Xiaotao Zhang, Xinrong Li, Xuankun Yang, Xun Zhang, Yan Li, Ye Lu, Yi Wang, Yibo Zhou, Yichi Zhang, Yihao Sun, Yijun Huang, Yixin Zhu, Yixuan Wu, Yuchen Sun, Yue Wu, Yuheng Sun, Yukun Li, Yutian Tu, Yuxuan Qin, Yuzhuo Wu, Zeyu Li, Zhengyu Lou, Zhenning Ran, Zizhu He, Pengfei Liu. (2026). AcademiClaw: When Students Set Challenges for AI Agents. arXiv preprint. https://arxiv.org/abs/2605.02661 doi:10.48550/arxiv.2605.02661

q2 · i? · associational · r?

Correlational analysis of trajectory logs from the six-model evaluation, using token counts recorded by an API logging proxy. The article interprets the absence of positive return as agents continuing past the point of diminishing returns; the printed r = −0.03 is the only effect-size-like statistic.

"pooling all 480 model–task evaluations, the Pearson correlation between token consumption and task score is 𝑟=−0.03 (𝑝=0.49 )—effectively zero. Even within individual models, no correlation exceeds|𝑟|=0.08"

Discussion¶

Related Claims¶

• Safety and task quality are largely independent across models, implying safety guardrails impose no measurable performance tax — related
• Professional developers write prompts of moderate baseline quality, which does not meaningfully scale with programming experience — related