Pragmatic and contingent LLM rewrites reduced perceived cognitive load relative to standard interpreter messages¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Across the full sample, intrinsic load was lower for Pragmatic vs. Standard (p=0.0017) and Contingent vs. Standard (p=0.0431), and extraneous load differed only for Pragmatic vs. Standard (p=0.0064). → Alexandru-Radu Moraru 2026 q2 i? For novices, extraneous load was lower for Pragmatic vs. Standard (p=0.039); for experts, Pragmatic vs. Standard showed lower intrinsic (p=0.0049) and higher germane load (p=0.0043). → Alexandru-Radu Moraru 2026 (2)

Evidence¶

Alexandru-Radu Moraru 2026¶

Alexandru-Radu Moraru, Shreyan Biswas, Ujwal Gadiraju. (2026). Beyond the Traceback: Using LLMs for Adaptive Explanations of Programming Errors. https://arxiv.org/abs/2608.20896 doi:10.48550/arxiv.2608.20896

q2 · i? · causal · r2

Kruskal-Wallis tests with Holm-adjusted Dunn pairwise comparisons on Likert ratings from the full sample of 103 participants in the main Prolific experiment; the article reports "differences inintrinsic loadfor Prag- matic vs. Standard (𝑝= 0.0017)".

"Forcognitive load, we observe differences inintrinsic loadfor Prag- matic vs. Standard (𝑝= 0.0017) and Contingent vs. Standard (𝑝= 0.0431)."

Alexandru-Radu Moraru 2026 (2)¶

Alexandru-Radu Moraru, Shreyan Biswas, Ujwal Gadiraju. (2026). Beyond the Traceback: Using LLMs for Adaptive Explanations of Programming Errors. https://arxiv.org/abs/2608.20896

q2 · i? · causal · r2

Skill-group analysis of the same experiment (38 novices, 65 experts); novices showed "lowerextraneous loadfor Pragmatic vs. Standard ( 𝑝= 0.039)", and experts showed lower intrinsic and higher germane load for Pragmatic vs. Standard.

"For novices, Holm-adjusted Dunn tests indicate lowerextraneous loadfor Pragmatic vs. Standard ( 𝑝= 0.039) and also for Contingent vs. Standard (𝑝= 0.054)."

Discussion¶

Related Claims¶

• Both LLM rewrite styles were rated less authoritative than standard interpreter messages — related
• LLM-rewritten Python error messages improved subjective evaluations but did not reliably improve objective debugging performance — related
• Descriptive time-to-fix patterns favored pragmatic messages, but statistical tests found no significant effects — related
• Students using ChatGPT 3.5 experience lower intrinsic cognitive load than students using Google during a web research task — related