Baseline capability moderates AI productivity benefits non-monotonically across AI's capability frontier¶

Claim · All claims
Evidence · 1 study · theoretical r3 · q1
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q1 i? Novice workers gain more from AI assistance within AI's reliable capability boundary, while for tasks outside that boundary reliance on AI degrades performance, potentially more for less experienced workers who cannot recognize unreliable AI outputs. → Won Ik Cho 2026

Evidence¶

Won Ik Cho 2026¶

Won Ik Cho, Seong-hun Kim, Geunhye Kim. (2026). Position: Adopting AI in Practice Does Not Guarantee the Productivity Boost. Proceedings of the 43rd International Conference on Machine Learning, PMLR 306. https://arxiv.org/abs/2605.24688 doi:10.48550/arxiv.2605.24688

q1 · i? · theoretical · r3

The paper cites two field experiments as its empirical grounding: a customer-support field experiment reporting gains up to 36% for the lowest-skilled workers, and a randomized experiment with management consultants showing workers using AI on out-of-frontier tasks were "19 percentage pointsless likely to be accuratethan those without AI access." The paper itself reports no new data.

"Brynjolfsson et al. (2025) found productivity gains of up to 36% for the lowest-skilled workers, with minimal impact on the most experienced staff, while Dell'Acqua et al. (2023) demonstrated that for tasks outside AI's capability frontier, workers using AI performed 19 percentage pointsless likely to be accuratethan those without AI access."

Discussion¶

Related Claims¶

• Between-group effect sizes on decision-making gains were large for AI-assisted versus traditional instruction and moderate-to-large for AI-assisted versus inquiry-only — related
• Within the AI-assisted group, the frequency of checking the AI's claims against sources predicted decision-making gains, with no students flagged for over-reliance — related
• Practitioners who adopt AI assistance show reduced independent performance compared to their baseline — a narrower finding that bears on this claim
• Study setting significantly moderates GenAI productivity effects: laboratory experiments show large significant gains (g = 0.73) while enterprise (g = 0.19) and open-source (g = 0.01) effects are substantially smaller and non-significant. — related
• AI Interaction Competence, not GPA or prior knowledge, is the decisive moderator of LLM-assisted learning gains — a narrower finding that bears on this claim
• GPT-4 access raised consultant output quality inside the model's frontier but reduced correct solutions just outside it — a narrower finding that bears on this claim
• Over a quarter of benchmark tasks (23 of 80) defeat all six frontier models, with over 22% showing capability boundaries where scores swing by up to 90 points — related
• In one randomized working-paper experiment, conceptual-map scaffolding gave LLM-using novices marginally higher scores (p < .10) and lower score variance, with a marginal benefit concentrated among low-AIC novices — related
• Sequencing independent task engagement before LLM consultation is associated with stronger student outputs — related