A 45-minute prompt-based programming lesson produced greater pre-to-post gains in Promptly problem solving than a code-tracing control, though not statistically significant¶

Claim · All claims
Evidence · 1 study · causal r2 · q3
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? Students in the experimental prompting lesson improved more on Promptly problem solving from pre- to post-test than the control group (B = 0.756, 95% CI [-0.020, 1.533], p = 0.056), an effect that was not statistically significant. → Keith Tran 2026

Evidence¶

Keith Tran 2026¶

Keith Tran, Samiha Marwan, and Thomas Price. (2026). Teaching Prompt-Based Programming with LLMs: A 45-Minute Lesson with Guided Practice for End-User Programmers. https://arxiv.org/abs/2606.30547 doi:10.48550/arxiv.2606.30547

q3 · i? · causal · r2

Randomized controlled pre-test/post-test study with 55 engineering students (28 experimental, 27 control) analyzed with linear mixed-effects models; the experimental group improved by an estimated 0.87 problems versus 0.110 for control, "an effect larger than that of a full standard deviation of prior programming experience."

"The interaction between Time and Condition suggests that students in the experimental group showed greater gains from pre- to post-test than the control group ( 𝐵 = 0.756, 95% CI [−0.020, 1.533],𝑝 = 0.056)."

Discussion¶

Related Claims¶

• The 45-minute prompting lesson significantly increased prompting self-efficacy relative to a business-as-usual control lesson — related
• When prompts failed, students most commonly reported clarifying intent or specifying more detail, and reflecting on the problem depiction, rather than tracing code or examining test cases — related
• The classroom training is associated with a greater DCCS flexibility gain in the experimental group, though the printed ANOVA table reports a non-significant interaction — related
• Prior programming experience significantly predicted Promptly task performance and prompting self-efficacy — related
• Pre-written prompts increase Justifications and Alternative ideas in children's pair-programming talk and shift the role of Simple questions — related