Checking can coexist with unresolved errors: 7 of 15 students failed a task containing plausible LLM falsehoods despite self-checking¶
Claim · All claims
Evidence · 1 study · reviewr2·q2
Currency · newest study 2026 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? In a think-aloud study of 15 students completing two tasks with plausible LLM falsehoods, seven students failed Task 1 despite self-checking or external cross-checking (46.7%; Wilson 95% CI, 24.8%–69.9%). → Wei J and Shang Y 2026
Evidence¶
Wei J and Shang Y 2026¶
Wei J and Shang Y (2026). Beyond checking: verification quality, reliance calibration, and learning in generative AI-assisted higher education. Front. Psychol. 17:1965371. https://doi.org/10.3389/fpsyg.2026.1965371
q2 · i? · review · r2
Mini Review; this entry is its summary (text and evidence table) of one reviewed study, Zhang et al. (2025), a think-aloud case study of 15 students using ChatGPT (GPT-3.5, Oct–Dec 2023) on two academic tasks with known plausible falsehoods; LLM self-checking and external cross-checking were observed and final accuracy scored. The review reports "Seven students failed Task 1 despite self-checking or external cross-checking".
"Zhang et al. (2025) asked 15 students to complete two tasks containing plausible LLM falsehoods while thinking aloud and using external sources. Seven students failed Task 1 despite self-checking or external cross-checking (46.7%; Wilson 95% CI, 24.8%–69.9%)."
Discussion¶
Related Claims¶
- Reviewed literature reports risks of overreliance, AI errors in complex tasks, and students' uncritical acceptance of AI-generated outputs — a broader claim this one bears on
- In a ten-request live textbook audit, citation integrity coexisted with substantive error: every citation resolved to its source, yet answers included retrieval misses and contradictions — related
- AI output errors are frequent: 44.5% fabricated references in drafted research proposals and 5–13% unsafe chatbot medical answers — related
- Generative AI makes knowledge mobilizable, converting external knowledge into immediately usable task output within seconds — related
- AI-generated ethics exercise materials carried external citations that did not survive checking, ranging from misattributed to apparently fabricated — related
- Experimental-group students detected AI errors and enacted verification and ethical self-regulation as routine practices — reports the opposite
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
checking-coexists-with-unresolved-error-zhang |
title |
Checking can coexist with unresolved errors: 7 of 15 students failed a task containing plausible LLM falsehoods despite self-checking |
description |
Checking can coexist with unresolved errors: 7 of 15 students failed a task containing plausible LLM falsehoods despite self-checking |
status |
review |
evidence_strength |
weak |
newest_evidence |
2026 |
source_checked |
2026-10-20 |
generated |
by process:wiki-ingest on 2026-10-11 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — wei-j-and-shang-y-2026 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.