GenAI coding assistance produces a statistically significant moderate positive effect on developer productivity (g = 0.33), with substantial heterogeneity across settings.¶

Claim · All claims
Evidence · 1 study · quant-synthesis r3 · q3 · i1 small
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i1 Across 16 effect sizes from 14 studies, GenAI assistance yields a statistically significant moderate pooled productivity gain (Hedges' g = 0.33, 95% CI [0.09, 0.58], p = 0.008). → Maier 2026 q3 i? Productivity effects vary substantially across studies (I² = 99%, τ² = 0.22, Q(15) = 206.06, p < 0.001), indicating gains depend strongly on context rather than being uniform. → Maier 2026

Evidence¶

Maier 2026¶

Maier, S., Gunzenhäuser, M., Schweisthal, J., Schneider, M., & Feuerriegel, S. (2026). A meta-analysis of the effect of generative AI on productivity and learning in programming. arXiv preprint. https://arxiv.org/abs/2605.04779 doi:10.48550/arxiv.2605.04779

q3 · i1 · quant-synthesis · r3

Random-effects meta-analysis (REML) pooling 16 effect size estimates from 14 studies comparing GenAI-assisted with unassisted programming. The article reports "a statistically significant, positive but moderate increase in productivity" (g = 0.33, 95% CI [0.09, 0.58], SE = 0.13, p = 0.008). Heterogeneity is substantial: I² = 99%, τ² = 0.22, Q(15) = 206.06, p < 0.001.

"Overall, we find a statistically significant, positive but moderate increase in productivity associated with GenAI assistance (Hedges’g= 0.33,95%confidence interval [CI]:[0.09,0.58], SE = 0.13,p= 0.008)."

Discussion¶

Related Claims¶

• GenAI assistance shows no statistically significant pooled effect on programming learning outcomes (g = 0.14, 95% CI (-0.18, 0.47)), with substantial heterogeneity. — related
• Study setting significantly moderates GenAI productivity effects: laboratory experiments show large significant gains (g = 0.73) while enterprise (g = 0.19) and open-source (g = 0.01) effects are substantially smaller and non-significant. — related
• Effects of AI literacy interventions vary considerably across contexts, with most variance between studies (71.361%) and a wide prediction interval — related
• Exam environment critically moderates GenAI learning effects: significant large gains when GenAI is permitted during assessment (g = 0.76) versus non-significant slightly negative effects when it is not (g = -0.06). — related