Prompting strategies differ by bug source: natural bugs elicit specification-oriented strategies while injected bugs elicit guidance-oriented strategies¶

Claim · All claims
Evidence · 1 study · qualitative r2 · q3
Currency · no dated study recorded · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? Natural bug states are dominated by specification-oriented strategies (Update signature 30.6%, Extend task 27.8% in n:first), while injected bug states show more guidance and restatement-oriented strategies (Provide guidance 33.0% in i:any). → rq2-strategies

Evidence¶

rq2-strategies¶

Victor-Alexandru Pădurean, Kaitlin Riegel, Alkis Gotovos, Jyotika Mahapatra, Ahana Ghosh, Paul Denny, Juho Leinonen, James Prather, and Adish Singla. 2026. When AI Is Wrong on Purpose: How Students Respond to Buggy GenAI Code. Proceedings of the ACM Conference on International Computing Education Research Vol. 1 (ICER 2026 Vol. 1). https://doi.org/10.1145/3765964.3811667

q3 · i? · qualitative · r2

Qualitative coding of 204 sampled prompts using a single-label coding scheme with Krippendorff's α = 0.87. The codebook includes eight strategy categories from Task reframe to Meta. Distribution differs by bug source across n:first, n:any, i:first, and i:any groups.

"Natural bug states are dominated by specification-oriented strategies, while injected bug states show more guidance and restatement-oriented strategies."

Discussion¶

Related Claims¶

• Immediate success rates are higher after injected bugs than after natural bugs — related
• Injected bugs prompt more direct code edits while natural bugs prompt more reprompting in prompt-based programming tasks — related
• Students make smaller code changes and fewer edit-and-run attempts after injected bugs than after natural bugs — related
• Students report code understanding, debugging, and AI-limitation awareness as primary learning benefits of buggy GenAI programming tasks — related