In a 700-student open-ended systems programming project graded by specification document rather than code, in-person test failure rates rose and inspected code failed to meet specifications¶

Claim · All claims
Evidence · 1 study · qualitative r1 · q1
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q1 i? After an open-ended final project graded via written specification documents, failure rates rose on the course's in-person tests, and the instructor judged students had deluded themselves about learning. → Muhammad Sajjad Akbar 2026

Evidence¶

Muhammad Sajjad Akbar 2026¶

Muhammad Sajjad Akbar, Geoffrey Challen, Fraida Fund, Casey Hopkins, Oscar Karnalim, Kevin Lin, James McGuffee, Shubbhi Taneja, Ranysha Ware. (2026). AI Can Do Your Homework. Now What? Report from an online workshop on computing assessment in the age of generative AI. https://arxiv.org/abs/2609.27842 doi:10.48550/arxiv.2609.27842

q1 · i? · qualitative · r1

Practitioner report of an open-ended final project in a 700-student systems programming course, graded on a written document explaining how specifications were met, with a build script and demo video. The instructor attributed rising in-person test failure to self-deception about learning, not cheating.

"Failure rates rose on the course's in-person tests, and the instructor's reading was not that students were cheating but that they “deluded themselves into thinking they were learning when they weren’t.”"

Discussion¶

Related Claims¶

• Algebra failure rates rose post-policy for above-norm students (~3 percentage points) but fell for below-norm students (~4 points); classroom environment and peer ability explained most of the above-norm rise but not the below-norm fall — related
• Students from socioeconomically disadvantaged backgrounds fail dual enrollment at higher rates than equally achieving peers, with the gap widening as achievement rises (the readiness illusion) — related