When prompts failed, students most commonly reported clarifying intent or specifying more detail, and reflecting on the problem depiction, rather than tracing code or examining test cases¶

Claim · All claims
Evidence · 1 study · qualitative r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Among 174 students coded for refinement strategies after unsuccessful B2 attempts, the most frequent strategy was clarifying intent or more thoroughly specifying details, followed by reflecting on the problem depiction; tracing incorrect code and reflecting on test cases/error messages were less likely to be reported. → Victor-Alexandru Pădurean 2026

Evidence¶

Victor-Alexandru Pădurean 2026¶

Victor-Alexandru Pădurean, Kaitlin Riegel, Gweneth Barbre, Musa Blake, Paul Denny, Alkis Gotovos, Juho Leinonen, Stephen MacNeil, James Prather, and Adish Singla. (2026). Understanding Student Perceptions, Mistakes, and Debugging Approaches when Solving Natural Language Programming Tasks. Proceedings of the ACM Conference on International Computing Education Research Vol.1 (ICER 2026 Vol. 1). https://doi.org/10.1145/3765964.3811639

q2 · i? · qualitative · r2

Thematic analysis of Ref4 responses (n = 174 coded for strategies) from students who made at least one mistake on B2 problems; Figure 5 ranks strategy frequencies. The article states clarifying intent was most frequent and that tracing code and reflecting on test cases were less reported.

"The most frequently identified strategy involved Clarifying intent or more thoroughly specifyingdetails in their prompt. Many students also indicated they would furtherRe- flect on problem depictionitself."

Discussion¶

Related Claims¶

• B2 problems required more effort than B1 problems, with lower success rates and greater variance in conversations, messages, and code executions — related
• The most common prompt mistakes are omissions of key details, most frequently the specified function return type — related
• A 45-minute prompt-based programming lesson produced greater pre-to-post gains in Promptly problem solving than a code-tracing control, though not statistically significant — related
• Strategy use showed no significant relationship with problems solved in B2, except writing code as prompt (r = .20), based on only twelve students — related
• When refined prompts failed, novices overgeneralized and concluded LLMs could not detect anaphors, attributing failure to the model rather than to their prompting — related
• Students most frequently reported that prompt-based tasks were Easier/Simpler/Faster and supported analyzing and solving problems, in thematic analysis of 200 reflections — related