Perfect AI execution cannot remove four limits: accuracy does not settle end-setting, safety does not confer legitimacy, detection does not secure integrity, and outputs do not replace participation¶

Claim · All claims
Evidence · 1 study (2 entries) · theoretical r3 · q1
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q1 i? Even a perfectly accurate AI system cannot make a target legitimate, because accuracy concerns means while end-setting concerns which purposes deserve authority. → Kai Yao 2026 q1 i? A system can be safe in the narrow sense and still lack legitimacy, because legitimacy asks whether a decision has standing and is answerable to affected people. → Kai Yao 2026 (2)

Evidence¶

Kai Yao 2026¶

Kai Yao. (2026). When AI Does the Work, What Is Learning For? Post-Instrumental Learning and the Risk of Capacity Dissolution. arXiv. https://arxiv.org/abs/2607.28041 doi:10.48550/arxiv.2607.28041

q1 · i? · theoretical · r3

Theoretical argument within the article's perfect-AI idealization. The article argues accuracy is an epistemic good about representing facts, but "an accurate map does not choose the destination", so end-setting remains a learned human capacity.

"Even a perfectly accurate model can carry forward a bad problem formulation. It can show which means best serve a target, but it cannot make the target legitimate by serving it well."

Kai Yao 2026 (2)¶

Kai Yao. (2026). When AI Does the Work, What Is Learning For? Post-Instrumental Learning and the Risk of Capacity Dissolution. arXiv. https://arxiv.org/abs/2607.28041

q1 · i? · theoretical · r3

Theoretical argument on the safety/legitimacy distinction. The article holds that transparency, explanation, human review, and recourse remain necessary in a world of more reliable AI because "legitimacy is not reducible to error correction".

"A system can be safe in the narrow sense and still lack legitimacy. It may produce correct benefit classifications while affected people have no meaningful way to challenge the rule."

Discussion¶

Related Claims¶

• Surface validation can function while substantive validation fails: most participants detected flawed AI recommendations yet continued following them — related
• Reviewed literature reports risks of overreliance, AI errors in complex tasks, and students' uncritical acceptance of AI-generated outputs — related
• Unclear policy, workload pressure, and lack of mechanisms to monitor student AI use undermined staff confidence and perceived fairness during AIAS implementation — related
• Educators struggled to reliably differentiate student-written from GenAI-generated reflections — related
• Participants articulated four interrelated arguments for preserving human instructor authority in assessment: knowledge of students, dialogue, institutional legitimacy, and safeguarding against AI error — a narrower finding that bears on this claim
• Transparency about AI involvement appeared to function as an instructional intervention, shifting student reflection from feedback quality toward authority, trust, and assessment legitimacy — a narrower finding that bears on this claim
• Source provenance labels functioned as a seal of authority: 11 of 12 participants failed to detect a planted source-card mismatch — a narrower finding that bears on this claim
• Oral assessments are recommended as a strategy to maintain academic integrity against AI use — related
• Generative AI platforms cannot yet independently lead meaningful innovation; at best they augment human intelligence for innovation — related
• Cultivating AI intuition safeguards against technological overreliance and erosion of critical thinking (theoretical argument) — related
• Holding a strong goal intention alone does not guarantee goal achievement — a narrower finding that bears on this claim
• Ungraded practice tests function as a legitimate mediating practice that empowers learners and redefines the instructor's authority — related
• AI-generated text can pass as human-authored in up to 80% of cases, making reliable differentiation a significant challenge — related
• Detection-oriented integrity approaches are increasingly misaligned with GenAI-augmented assessment (theoretical argument) — related
• Every assessment procedure or instrument embodies a sense of telos, producing data that locate people along the researcher's assumed developmental path — related