A 60-minute session with Prompt Coach significantly improved professional developers' prompt quality, a 13.73% relative gain overall¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r1 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? After a 60-minute PC learning session, mean prompt-quality scores rose from 63.04 to 71.69 (+13.73%, paired Wilcoxon p < .001). → Rohit Mehra 2026 q2 i? 13 of 15 participants improved and no participant showed a meaningful decline. → Rohit Mehra 2026 (2)

Evidence¶

Rohit Mehra 2026¶

Rohit Mehra, Kapil Singi, Vikrant Kaulgud, Vibhu Saujanya Sharma, Swapnajeet Gon Choudhury, Swati Sharma, Adam P. Burden, Majd Sakr. (2026). Prompt Coach: An Empirical Evaluation of an Agentic Tutor for Learning Prompt Engineering in Software Development. https://arxiv.org/abs/2607.06074 doi:10.48550/arxiv.2607.06074

q2 · i? · design · r2

Single-arm, within-subjects pre/post study of 15 professional developers (mean 9.6 years experience); prompts scored via LLM-as-judge across eight dimensions. Table 2 reports the mean rise and paired Wilcoxon p < .001, described as "a 13.73% relative improvement over the baseline".

"After a 60-minute learning session with PC, post-learning prompts achieved a mean score of 71.69, representing a 13.73% relative improvement over the baseline"

Rohit Mehra 2026 (2)¶

Rohit Mehra, Kapil Singi, Vikrant Kaulgud, Vibhu Saujanya Sharma, Swapnajeet Gon Choudhury, Swati Sharma, Adam P. Burden, Majd Sakr. (2026). Prompt Coach: An Empirical Evaluation of an Agentic Tutor for Learning Prompt Engineering in Software Development. https://arxiv.org/abs/2607.06074

q2 · i? · causal · r1

Participant-level results from the same pre/post study: "13 out of 15 participants showed improvements"; the small negative change (-0.28%) is attributed by the authors to variability in LLM-as-a-judge scoring.

"Notably, 13 out of 15 participants showed improvements, while 2 participants exhibited no measurable gain on average. Importantly, no participant showed a meaningful decline in performance."

Discussion¶

Related Claims¶

• Post-test mathematics performance was significantly lower than pre-test after AI-tutor use — related
• Professional developers write prompts of moderate baseline quality, which does not meaningfully scale with programming experience — related
• Prompt Coach's largest learning gains occurred on Inclusion of Constraints, Error Handling, and Context Awareness, the dimensions developers scored lowest on at baseline — related
• Students in a scaffolded LLM-collaboration course improved significantly on initial argument quality, prompt engineering, and response to AI feedback from the first to the fifth assignment — related
• A professional development masterclass produces a 24% overall improvement in educators' AI utilization competence with large effect sizes — related