GenAI undermines evidence rules by making work products ambiguous between student competence and AI capability, with substitution capacity depending on task cognitive demands¶

Claim · All claims
Evidence · 1 study (2 entries) · theoretical r3 · q1
Currency · newest study 2026 · checked against its sources 2026-10-20 (automated, 2 of 2 entries) · not reviewed by a person

Subclaims¶

q1 i? GenAI contribution to work products breaks the inferential link from product features to student proficiency. → Gao 2026 q1 i? GenAI substitution is nearly complete for recall and algorithmic tasks but more limited for disciplinary judgment on novel or ambiguous problems. → Gao 2026 (2)

Evidence¶

Gao 2026¶

Gao, Y., Chen, Z., Li, M., & Zhai, X. (2026). Generative AI as a Design Variable: An Evidence-Centered Framework for Principled Governance in STEM Assessment. https://arxiv.org/abs/2605.24837 doi:10.48550/arxiv.2605.24837

q1 · i? · theoretical · r3

Theoretical analysis within the paper's ECD framework, not an empirical study. The authors argue that when GenAI contributes to products, "it becomes unclear whether they reflect student competence or AI capability," so correctness scores are no longer interpretable as indicators of the targeted construct.

"GenAI complicates the evidence model by undermining evidence rules—the procedures that map observed student work products to latent constructs—because when GenAI contributes to those products, it becomes unclear whether they reflect student competence or AI capability."

Gao 2026 (2)¶

Gao, Y., Chen, Z., Li, M., & Zhai, X. (2026). Generative AI as a Design Variable: An Evidence-Centered Framework for Principled Governance in STEM Assessment. https://arxiv.org/abs/2605.24837. doi:10.48550/arxiv.2605.24837

q1 · i? · theoretical · r3

Authors' analytical claim about the dependence of GenAI substitution on task cognitive demands, supported by cited prior findings (e.g., ChatGPT narrowly passing an introductory physics course per Kortemeyer, 2023) rather than by new data in this paper.

"for tasks requiring factual recall or standard algorithmic problem solving, GenAI can substitute for student cognition almost completely; for tasks requiring disciplinary judgement applied to novel or ambiguous problems, its substitution capacity is more limited"

Discussion¶

Related Claims¶

• In TBLT, teachers perceive scaffolding collapse, a shortcut paradox, invisible inequity, and task realism corrosion — related
• The review reports some evidence that over-dependence on AI contributes to cognitive offloading and decreased independent reasoning, while guided AI use can be helpful — related
• AI assistance that substitutes for learning activities may improve immediate performance while impeding long-term skill development — related
• The article reports observational patterns that AI assistance can improve short-term performance while reducing cognitive engagement and independent capability — related
• Students' generative AI use may function either as a scaffold supporting thinking or as a shortcut bypassing essential cognitive processes — related
• Generative AI weakens the evidentiary relation between assessment artifacts and learner capacity, so institutions must assess the learner's accountable relation to AI-mediated work — possibly the same claim (merge candidate)
• Substituting tasks with generative AI may promote inclusion and free time for more demanding activities, but risks foundational-skill loss and presupposes prior knowledge — related
• Proposition 4: evidence gathered through distinct demands supports capability claims better than multiple records of the same performance — related
• Algorithmically inferred measurement functions may intensify, rather than resolve, the problem of nomic measurement in analyzing student learning — related
• Students in an academic writing course developed a more nuanced understanding of AI's capabilities and limitations, engaging with it as a support tool rather than a substitute for original work — a narrower finding that bears on this claim