Oral and explanation-based evaluation of AI-leveraged projects is complicated because students may break down on counterfactual questions even when they did the work¶

Claim · All claims
Evidence · 1 study · qualitative r2 · q1
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q1 i? In AI-leveraged project courses, a student's inability to explain a system may reflect weak understanding, weak communication skills, or both, and participants observed students failing counterfactual questions despite having done the work. → Muhammad Sajjad Akbar 2026

Evidence¶

Muhammad Sajjad Akbar 2026¶

Muhammad Sajjad Akbar, Geoffrey Challen, Fraida Fund, Casey Hopkins, Oscar Karnalim, Kevin Lin, James McGuffee, Shubbhi Taneja, Ranysha Ware. (2026). AI Can Do Your Homework. Now What? Report from an online workshop on computing assessment in the age of generative AI. https://arxiv.org/abs/2609.27842 doi:10.48550/arxiv.2609.27842

q1 · i? · qualitative · r2

Workshop discussion in the Ambitious AI-Leveraged Projects room about evaluation practices combining deliverables, presentations, oral examinations, and counterfactual questioning. Participants report the distinction between weak understanding and weak communication has become harder to draw.

"a student’s inability to explain a system may reflect weak understanding, weak communication skills, or both—a distinction some participants observed has become harder, with students breaking down on counterfactual questions even when they had done the work."

Discussion¶

Related Claims¶

• Computing educators' interest in AI-era assessment adaptation was spread roughly evenly across seven strategies, dividing between redesigning what is assigned and redesigning how work is verified — related
• Student narratives complicated the educator assumption that copied AI answers imply no learning, revealing invisible post-AI learning activities — related
• Students describe concrete verification behaviors for GenAI explanations, treating evaluation as an iterative filter rather than a single accept-or-reject judgment — related