Boundary compliance (S3) drives nearly all inter-model safety divergence, with a 53-point gap between the safest and least safe models¶

Claim · All claims
Evidence · 1 study · design r? · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? Four of five safety dimensions show modest inter-model variation, but S3 (boundary compliance) exhibits a 53-point chasm: the two Claude models lead at 83–85 while Gemini (31.6) and Qwen3.5 (34.4) fall substantially behind. → Junjie Yu 2026

Evidence¶

Junjie Yu 2026¶

Junjie Yu, Pengrui Lu, Weiye Si, Hongliang Lu, Jiabao Wu, Kaiwen Tao, Kun Wang, Lingyu Yang, Qiran Zhang, Xiuting Guo, Xuanyu Wang, Yang Wang, Yanjie Wang, Yi Yang, Zijian Hu, Ziyi Yang, Zonghan Zhou, Binghao Qiang, Borui Zhang, Chenning Li, Enchang Zhang, Feifan Chen, Feng Jian, Fengyin Sun, Hao Qiu, Hao Zheng, Haoran Zhu, Hongyu Liu, Jianbin Deng, Jiaxin Song, Jiaying Chi, Jiayou Shi, Jie Fang, Jinghui Zhong, Jingyu Zhou, Jinze Li, Junfeng Yi, Junyan Yu, Junzhi Xue, Ni Song, Pengyi Chen, Qi Chen, Quansheng Li, Rui Tao, Shenghai Gong, Shenhang Lu, Tianqi Shen, Tianxiang Zhu, Tiehan Kang, Tingyu Li, Wendi Wu, Xiao Shen, Xiao Zhou, Xiaotao Zhang, Xinrong Li, Xuankun Yang, Xun Zhang, Yan Li, Ye Lu, Yi Wang, Yibo Zhou, Yichi Zhang, Yihao Sun, Yijun Huang, Yixin Zhu, Yixuan Wu, Yuchen Sun, Yue Wu, Yuheng Sun, Yukun Li, Yutian Tu, Yuxuan Qin, Yuzhuo Wu, Zeyu Li, Zhengyu Lou, Zhenning Ran, Zizhu He, Pengfei Liu. (2026). AcademiClaw: When Students Set Challenges for AI Agents. arXiv preprint. https://arxiv.org/abs/2605.02661 doi:10.48550/arxiv.2605.02661

q2 · i? · design · r?

Five-category rule-based safety audit of agent tool-call trajectories across the six-model evaluation (Table 6). Gemini and Qwen3.5 accumulated 217 and 146 HIGH-severity violations respectively, predominantly file access outside the designated workspace.

"four of five safety dimensions show modest inter-model variation (S1: 85–95; S2: 87–90; S4: 90–98; S5: 73–83), but S3 (boundary compliance) exhibits a 53-point chasm."

Discussion¶

Related Claims¶

• Inter-model agreement on LLM skill tagging is moderate (Cohen's kappa 0.51), with strong agreement on S5 and disagreement concentrated in the S2/S3/S4 boundary — related
• LLM attack success rates increase consistently from single-turn to static multi-turn and dynamic multi-turn interactions in educational settings — related
• No evaluated LLM tutor is reliably safe: every model exceeds 60% harm rate on at least five risk categories in single-turn and six in multi-turn evaluation — related
• Educational safety performance varies widely across LLMs, with GPT-5.6-Luna the safest and Qwen3-8B among the weakest under educational adversarial probing — related
• AI teaching agents differed most on knowledge-related dimensions and least on role-play quality, with adaptive difficulty calibration a shared weakness — related