Recommendation homogeneity does not differ between students using ChatGPT 3.5 and students using Google¶
Claim · All claims
Evidence · 1 study · causalr2·q3
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q3 i? Contrary to the hypothesis, students using LLMs did not show less variation in their recommendations than students using traditional search engines (χ²(2) = 1.78; p = 0.411); the distribution of recommendations was similar for the two groups. → Matthias Stadler 2024
Evidence¶
Matthias Stadler 2024¶
Matthias Stadler, Maria Bannert, Michael Sailer. (2024). Cognitive ease at a cost: LLMs reduce mental effort but compromise depth in student scientific inquiry. Computers in Human Behavior. https://doi.org/10.1016/j.chb.2024.108386
q3 · i? · causal · r2
Chi-squared test on the frequency distribution of recommendations (against/neutral/in favor) across the two conditions in the randomized experiment (n=91; e.g., LLM: 9 against, 7 neutral, 28 in favor). No effect size is printed, so impact is null; equivalence was not tested beyond this test.
"Contrary to H3, students using LLMsdid not show less variation in theirrecommendations than students using traditional search engines (χ2(2)=1.78;p=0.411)."
Discussion¶
Related Claims¶
- Students using Google produce higher-quality justifications in their final recommendations than students using ChatGPT 3.5 — related
- ChatGPT users showed a non-significant trend toward worse overall informal learning outcomes than Google Search users over 8 days — related
- Students using ChatGPT 3.5 experience lower germane cognitive load than students using Google, the strongest of the three load differences — related
- Students using ChatGPT 3.5 experience lower extraneous cognitive load than students using Google during a web research task — related
- ChatGPT users differentiated information less (never by credibility) and rarely verified information compared to Google users — related
- Students using ChatGPT 3.5 experience lower intrinsic cognitive load than students using Google during a web research task — related
- Germane cognitive load fully mediates the effect of information-search tool on justification quality (exploratory analysis) — related
- LLM-supported socio-scientific inquiry lowered cognitive load but produced lower-quality reasoning and argumentation than conventional search — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
llm-recommendation-homogeneity-no-difference |
title |
Recommendation homogeneity does not differ between students using ChatGPT 3.5 and students using Google |
description |
Recommendation homogeneity does not differ between students using ChatGPT 3.5 and students using Google |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2024 |
generated |
by process:wiki-ingest on 2026-10-15 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — matthias-stadler-2024 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.