Diverse LLM explanations were associated with about 7.7% higher open-ended response accuracy than generic explanations, but the condition effect was not significant after Bonferroni adjustment¶

Claim · All claims
Evidence · 1 study · causal r2 · q3
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? First-year computing students who received three diverse LLM-generated explanations scored consistently higher on open-ended comprehension questions (about 7.7–8.1% across three snippets) than students who received generic explanations, but the condition effect did not remain significant after Bonferroni adjustment. → Seth Bernstein 2026

Evidence¶

Seth Bernstein 2026¶

Seth Bernstein, Paul Denny, Juho Leinonen, Kush Patel, Rayhona Nasimova, Matt Littlefield, and Stephen MacNeil. (2026). Exploring the Value of Diverse LLM Explanations in Introductory Programming. SIGCSE Virtual '26. https://arxiv.org/abs/2606.28882 doi:10.48550/arxiv.2606.28882

q3 · i? · causal · r2

Between-subjects randomized study of 971 first-year computing students assigned to diverse or generic GPT-4o explanations; OE responses from 785 students were manually coded and compared with chi-square tests and logistic regression. The article reports "open-ended response accuracy was consistently about 7.7% higher" for diverse explanations.

"Across participants, open-ended response accuracy was consistently about 7.7% higher when students received diverse explanations, with no difference in perceived cognitive load."

Discussion¶

Related Claims¶

• Multiple representations improve learning — the general claim this finding bears on, with the studies behind it
• Residual analysis of perception themes showed DD students more likely to report Engaged Understanding and less likely to report Overload/Frustration, though chi-square tests were not significant — related
• MCQ accuracy was high (above 87%) across all conditions with no significant difference between diverse and generic explanations — related
• DD students were less likely to dismiss explanations while GG students were more likely to dismiss them, with no statistically significant differences in use themes — related
• Students reported similar perceived information quantity across conditions, indicating diverse explanations did not produce perceived information overload — related