Strategy use showed no significant relationship with problems solved in B2, except writing code as prompt (r = .20), based on only twelve students¶
Claim · All claims
Evidence · 1 study · associationalr1·q2·i2medium
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i2 For each reported strategy there was no significant relationship with problems solved in B2, except Writing code as prompt (r = .20, p < .05), which the authors caution should be treated cautiously as only twelve students used it. → Victor-Alexandru Pădurean 2026
Evidence¶
Victor-Alexandru Pădurean 2026¶
Victor-Alexandru Pădurean, Kaitlin Riegel, Gweneth Barbre, Musa Blake, Paul Denny, Alkis Gotovos, Juho Leinonen, Stephen MacNeil, James Prather, and Adish Singla. (2026). Understanding Student Perceptions, Mistakes, and Debugging Approaches when Solving Natural Language Programming Tasks. Proceedings of the ACM Conference on International Computing Education Research Vol.1 (ICER 2026 Vol. 1). https://doi.org/10.1145/3765964.3811639
q2 · i2 · associational · r1
Correlational analysis linking coded refinement strategies (Ref4) to the number of B2 problems solved; the only significant association was "Writing code as prompt" with "r = .20", p < .05, a small-to-moderate printed correlation based on twelve students.
"Notably, for each strategy, there was no significant relationship with problems solved in B2, except forWriting code as prompt(r = .20,p< .05), which should be treated cautiously given that only twelve students were coded using this."
Discussion¶
Related Claims¶
- B2 problems required more effort than B1 problems, with lower success rates and greater variance in conversations, messages, and code executions — related
- Correlations between the number of prompt mistakes and number of messages sent were non-significant or weak, giving little evidence that mistake count influenced prompting effort — related
- When prompts failed, students most commonly reported clarifying intent or specifying more detail, and reflecting on the problem depiction, rather than tracing code or examining test cases — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
writing-code-as-prompt-strategy-correlates-solving |
title |
Strategy use showed no significant relationship with problems solved in B2, except writing code as prompt (r = .20), based on only twelve students |
description |
Strategy use showed no significant relationship with problems solved in B2, except writing code as prompt (r = .20), based on only twelve students |
status |
review |
evidence_strength |
weak |
newest_evidence |
2026 |
generated |
by process:wiki-ingest on 2026-10-11 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — victor-alexandru-pădurean-2026 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.