ChatGPT users showed a non-significant trend toward worse overall informal learning outcomes than Google Search users over 8 days¶

Claim · All claims
Evidence · 1 study · causal r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Participants using ChatGPT scored lower on average on the post-learning knowledge test than Google Search users, but the difference did not reach statistical significance. → Shravika Mittal 2026

Evidence¶

Shravika Mittal 2026¶

Shravika Mittal, Su Lin Blodgett, Q. Vera Liao. (2026). Learning by Chatting? Investigating the Impact of Generative AI on Information Seeking and Learning. https://arxiv.org/abs/2606.11669 doi:10.48550/arxiv.2606.11669

q2 · i? · causal · r2

Welch's t-test comparing post-learning knowledge test scores in an 8-day between-subjects field experiment (14 ChatGPT vs 21 Google completers). ChatGPT mean 11.62 (SD 2.32) vs Google 12.86 (SD 2.21); "did not reach statistical significance"; regression controlling for baseline showed a similar trend.

"The test did not reach statistical significance but indicates a trend that participants in the ChatGPT condition had worse learning outcomes than those in the Google condition (t-statistic:−1.523,p:0.069)."

Discussion¶

Related Claims¶

• Students using Google produce higher-quality justifications in their final recommendations than students using ChatGPT 3.5 — the general claim this finding bears on, with the studies behind it
• ChatGPT biased information access toward solution-oriented artifacts over principle-oriented information — related
• ChatGPT users differentiated information less (never by credibility) and rarely verified information compared to Google users — related
• ChatGPT users were significantly less likely to correctly answer a critical-learning question requiring interpretation of a nutrition label — related
• ChatGPT users were significantly less likely to correctly compare and evaluate animal and plant protein sources — related
• ChatGPT users began information seeking with narrow, specified queries far more often than Google users — related
• Recommendation homogeneity does not differ between students using ChatGPT 3.5 and students using Google — related
• Students using ChatGPT 3.5 experience lower extraneous cognitive load than students using Google during a web research task — related
• Google Trends data show ChatGPT search interest surged globally within weeks of its November 2022 public release, with plagiarism among the top related queries — related
• LearnMate2 improved learning outcomes and user experience compared to a baseline combining an online learning platform and an LLM in a user study (n=16) — related
• ChatGPT participants experienced meta-cognitive load and frictions, with slightly more unsatisfied endings than Google participants — related
• Access to web-based information creates illusions of learning and reduces memorization — related