Correlations between the number of prompt mistakes and number of messages sent were non-significant or weak, giving little evidence that mistake count influenced prompting effort¶
Claim · All claims
Evidence · 1 study · associationalr1·q2·i0negligible
Currency · newest study 2026 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i0 Pearson correlations between number of mistakes and number of messages were non-significant (B1-2 r = .01, B1-3 r = -.02, B2-3 r = .10) or weak (B1-1 r = .19, B2-1 r = .17, B2-2 r = .17), indicating little evidence that the number of mistakes influenced the number of prompts needed. → Victor-Alexandru Pădurean 2026
Evidence¶
Victor-Alexandru Pădurean 2026¶
Victor-Alexandru Pădurean, Kaitlin Riegel, Gweneth Barbre, Musa Blake, Paul Denny, Alkis Gotovos, Juho Leinonen, Stephen MacNeil, James Prather, and Adish Singla. (2026). Understanding Student Perceptions, Mistakes, and Debugging Approaches when Solving Natural Language Programming Tasks. Proceedings of the ACM Conference on International Computing Education Research Vol.1 (ICER 2026 Vol. 1). https://doi.org/10.1145/3765964.3811639
q2 · i0 · associational · r1
Secondary Pearson correlation analysis between number of messages and number of coded mistakes per problem among initially incorrect prompts. Printed coefficients include negligible values such as "r= .01" and weak values up to r = .19; three were non-significant.
"The results were non-significant, B1-2 (r= .01), B1-3 (r= -.02), and B2-3 (r= .10), or weak, B1-1 (r= .19,p< .05), B2-1 (r= .17,p< .01), and B2-2 (r= .17,p< .05), indicating little evidence to support the number of mistakes influencing the number of prompts needed."
Discussion¶
Related Claims¶
- B2 problems required more effort than B1 problems, with lower success rates and greater variance in conversations, messages, and code executions — related
- Strategy use showed no significant relationship with problems solved in B2, except writing code as prompt (r = .20), based on only twelve students — related
- The most common prompt mistakes are omissions of key details, most frequently the specified function return type — related
- Clusters did not differ significantly in the number of prompts sent to ChatGPT — related
- Missing prompt-element types significantly co-occur: failure to provide appropriate functionality explanation correlates strongly with missing expected output/return and with extraneous elements — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
mistake-count-message-count-weak-correlation |
title |
Correlations between the number of prompt mistakes and number of messages sent were non-significant or weak, giving little evidence that mistake count influenced prompting effort |
description |
Correlations between the number of prompt mistakes and number of messages sent were non-significant or weak, giving little evidence that mistake count influenced prompting effort |
status |
review |
evidence_strength |
weak |
newest_evidence |
2026 |
source_checked |
2026-10-20 |
generated |
by process:wiki-ingest on 2026-10-11 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — victor-alexandru-pădurean-2026 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.