Both LLM rewrite styles were rated less authoritative than standard interpreter messages¶

Claim · All claims
Evidence · 1 study · causal r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? In the full sample, both Pragmatic and Contingent messages differed from Standard on perceived authoritativeness (p=0.0001 and p=0.0032 respectively), with LLM styles rated less authoritative. → Alexandru-Radu Moraru 2026

Evidence¶

Alexandru-Radu Moraru 2026¶

Alexandru-Radu Moraru, Shreyan Biswas, Ujwal Gadiraju. (2026). Beyond the Traceback: Using LLMs for Adaptive Explanations of Programming Errors. https://arxiv.org/abs/2608.20896 doi:10.48550/arxiv.2608.20896

q2 · i? · causal · r2

Kruskal-Wallis with Holm-adjusted Dunn comparisons on a single tone item (whether the message felt respectful and reader-centered) from the main Prolific experiment (N=103); "both Pragmatic and Contin- gent differ from the Standard error message style".

"Finally, in terms ofauthoritativeness, both Pragmatic and Contin- gent differ from the Standard error message style (𝑝= 0.0001and 𝑝= 0.0032, respectively)."

Discussion¶

Related Claims¶

• LLM-rewritten Python error messages improved subjective evaluations but did not reliably improve objective debugging performance — related
• Pragmatic and contingent LLM rewrites reduced perceived cognitive load relative to standard interpreter messages — related
• Students hold cautious trust in the course-specific RAG system, expressing some level of trust but not treating it as an authoritative source and indicating they would verify important information — related
• Descriptive time-to-fix patterns favored pragmatic messages, but statistical tests found no significant effects — related