One-session checklist (CRAAP) instruction in source evaluation did no better than the alternative methods it was compared with in two quasi-experiments¶

Claim · All claims
Evidence · 3 studies · 2 causal r1, 1 qualitative r? · q2
Currency · newest study 2026 · 1 of 3 entries checked against their sources (automated) · not reviewed by a person

In the two quasi-experiments recorded here, a single library session teaching the CRAAP checklist did no better than the alternative it was compared with: six journalistic question words (Lowe et al. 2021) or a networked, lateral-reading-style framework (Ratcliffe 2026). In Ratcliffe (2026) the CRAAP group's accuracy rose from pre-test to a five-week post-test, with no untrained control group, so neither study shows checklist instruction failing outright. A third study (Wineburg & McGrew 2019) shows why on-page cues can mislead evaluators but taught no checklist.

Subclaims¶

q2 i? An expert/novice think-aloud study found that people who judge a website by its own features ("vertical" reading, as checklists direct) are misled by logos and domain names, while fact checkers who leave the site reach better-warranted conclusions faster. This is indirect evidence: it describes on-page evaluation and did not test checklist instruction. → Wineburg and McGrew 2019

q2 i? In a one-shot library session for first-year business students, CRAAP-checklist sections did no better or worse than six-question-word sections on source-evaluation quizzes, but produced lower-rated source use in end-of-semester papers. Students in both conditions still leaned on domain-name heuristics for web sources. This qualifies the claim: it is a comparison with a similar method, not with lateral reading. → Lowe et al. 2021

q2 i? Against the claim: in one-shot instruction with undergraduate nursing students, the CRAAP Test group's mean accuracy across the test sources (health-misinformation sites, reliable consumer-health sites and peer-reviewed articles) rose significantly from 41% at pre-test to 59% at a five-week post-test. It did not differ from the Source + Beyond the Source framework, a networked framework that includes lateral reading among its strategies, at any time point. There was no untrained control group. → Ratcliffe 2026

Evidence¶

Wineburg and McGrew 2019¶

Wineburg, S., & McGrew, S. (2019). Lateral reading and the nature of expertise: Reading less and learning more when evaluating digital information. Teachers College Record: The Voice of Scholarship in Education, 121(11), 1–40. doi:10.1177/016146811912101102

q2 · expert/novice think-aloud comparison (descriptive) · i? · no effect size reported · n=45 participants · qualitative · r?

Ten PhD historians, ten professional fact checkers and 25 Stanford undergraduates thought aloud while they evaluated live websites on social and political issues. Historians and students tended to stay on the site they were judging, and were taken in by easily manipulated features such as official-looking logos and domain names. Fact checkers left the site quickly and read laterally, and reached better-warranted judgments in less time. The study explains why on-page criteria can fail on the open web. It did not teach anyone a checklist, so it does not directly show that checklist instruction fails. Read from the abstract only.

Lowe et al. 2021¶

Lowe, M. S., Macy, K. V., Murphy, E., & Kani, J. (2021). Questioning CRAAP. Journal of the Scholarship of Teaching and Learning, 21(3), 33–48. doi:10.14434/josotl.v21i3.30744

q2 · quasi-experimental comparison of course sections · i? · effect sizes reported inconsistently · n=85 students (quizzes), 88 papers · causal · r1

A librarian taught first-year business-seminar sections one source-evaluation method in a single session: the CRAAP checklist or the six journalistic question words (who, what, when, where, why, how). Students were assessed with pre-, post- and end-of-semester quizzes and a paper rubric. Quiz scores and the quality of students' evaluation reasoning did not differ significantly between methods. End-of-semester papers from the question-word sections scored higher on the rubric (2.81 against 2.28 out of 4, p < 0.005). Many students in both groups still treated a site's domain as a sign of credibility and found the web source hard to evaluate. The paper reports Cohen's d = 0.10 for the rubric difference and calls it "large", which does not match its own F statistic, so the size of the effect is treated here as not established. The authors also note that the paper difference may come from more prior library instruction in the question-word group.

Ratcliffe 2026¶

Ratcliffe, C. (2026). Teaching source evaluation to undergraduate nursing students: Comparing the CRAAP Test and the Source + Beyond the Source framework. College & Research Libraries, 87(5), 555–580. doi:10.5860/crl.87.5.555

q2 · quasi-experimental comparison of course sections, pre/post · i? · no effect size reported (t-test p-values only) · n=115 students at pre-test (101 and 89 at the two post-tests) · causal · r?

First-term nursing students in 12 lab sections had one 110-minute library session. Half the sections were taught the CRAAP checklist and half a networked framework that sends students beyond the source. Students then rated the reliability of live health-misinformation sites, reliable consumer-health sites and peer-reviewed articles. Mean accuracy in the CRAAP group rose from 41% at pre-test to 52% immediately after and 59% five weeks later (pre-test to five weeks, p < .001). The two groups did not differ at any time point (p = .16, .49, .77). Both groups used surface heuristics such as design and domain less after training, but still used them. There was no untrained control group, and the author concludes that the one-shot format, rather than the checklist itself, limits what either method achieves. This result runs against the claim, at least in a one-shot library session.

Discussion¶

What the evidence below shows. None of the three studies shows checklist instruction failing outright. Wineburg & McGrew (2019) show why on-page evaluation misleads, but taught no checklist. Lowe et al. (2021) found no difference on quizzes between a CRAAP session and an alternative. Ratcliffe (2026) found accuracy after a single CRAAP session rose from 41% to 59% five weeks later, no different from a lateral-reading-style framework. The defensible reading is narrower than the title: one-session checklist instruction does no better than the alternatives it has been tested against, and on-page cues mislead evaluators. The paragraphs below predate this evidence. They are the standard argument against checklists, not findings recorded here; statements in them that no entry on this page supports are marked (unsourced on this page).

The mechanism of failure. The core problem is a mismatch between what checklists train and what skilled evaluators actually do. Checklist approaches direct attention inward, to features of the source itself (author credentials, domain suffix, "about" page, currency of the page), while proficient evaluators of online content tend to leave the page and read laterally — opening new tabs to see what other sites say about the source. Because checklist use keeps learners on the page under evaluation, it can be actively counterproductive (unsourced on this page; Lowe et al. 2021 found lower-rated source use in papers, against a similar method, not against no instruction): a polished, professional-looking page scores well on every checklist criterion while being entirely unreliable, and checklist training can inflate learners' confidence without improving their judgments (unsourced on this page). This confidence-without-competence pattern parallels other cases where surface cues mislead learners, as in Coherence principle: irrelevant material hurts learning — polished design is itself an irrelevant cue that checklists reward rather than discount.

Why checklists fail on the open web. Checklist criteria assume a relatively stable, gatekept publishing environment (print-era heuristics: publisher, author credentials, date). On the open web, low-quality and deceptive sources routinely satisfy every on-page criterion — professional design, an "about" page, citations, a plausible domain — so the checklist's surface cues are not diagnostic of reliability. The procedure also consumes working memory and instructional time on a rubric that adds procedural overhead without adding evaluative power (unsourced on this page), a load cost with no payoff in the same sense as Chunking reduces working memory load describes productive load reduction.

Constructive counterpart. This claim should be read alongside its alternative: instruction that models the strategies professional fact-checkers use — lateral reading, click restraint, and searching for prior coverage or fact-checks — does improve online evaluation. See Civic online reasoning instruction improves evaluation. The practical implication is not to abandon evaluation instruction but to replace checklist-based rubrics with strategy-based instruction grounded in authentic web tasks, consistent with Authentic audiences improve student work in that evaluation skill develops through real web contexts rather than decontextualized rubrics.

Boundary conditions and open questions. Most of the supporting research base comes from civic online reasoning contexts (news, websites, social media) with secondary and post-secondary learners; how checklist training interacts with domain expertise, younger learners, or curated (non-open-web) source sets remains underexplored. In curated collections where on-page metadata is reliable (e.g., a library database), checklist-style criteria may do less harm. Checklists may also retain limited value as a post-hoc reflection aid after lateral investigation, rather than as the primary evaluation procedure — a role closer to structured reflection tools such as 3-2-1 reflection than to a decision procedure.

Learner Variables¶

• Critical Thinking — outcome: instruction changes it

Related Claims¶

• AI literacy interventions show a large, statistically significant overall positive effect on student AI literacy outcomes (g = 0.837) — the general claim this finding bears on, with the studies behind it

• Civic online reasoning instruction improves evaluation — the strategy-based alternative that replaces checklists with lateral reading and fact-checker techniques

• Chunking reduces working memory load — checklists add procedural overhead without adding evaluative power, a load cost with no payoff
• Coherence principle: irrelevant material hurts learning — surface features like professional design are irrelevant cues that checklists mistakenly treat as diagnostic
• Authentic audiences improve student work — evaluation instruction works best grounded in authentic open-web tasks rather than decontextualized rubrics
• A medium feature's affordance does not guarantee its actual cognitive effects: LOGO instruction designed to enhance conditional reasoning may not do so — related
• Human expert review is the most credible but slowest and most resource-intensive evaluation method — related
• Computing students hold persistent incorrect beliefs, basing trust judgments on cues no longer reliable to guarantee trustworthiness — related
• Humanizer legitimacy performances show leakage: unendorsed logos, inactive social media, and no registered business address — related
• Reflective strategies such as cognitive forcing, pre-commitment, and uncertainty communication reduce overreliance on AI and improve decision quality, but have been evaluated only as short-term single-session interventions — related
• After the modules, students engaged in lateral reading far more often, rising from 3 of 87 students at pretest to 67 of 87 at posttest — related
• Students' reliance on surface-level evaluation strategies declined after the modules, including trust in dot-org domains, links, appearance, and About pages — related
• The intervention improved students' sourcing in credibility judgments and in written products relative to controls — related
• Gains in source evaluation held regardless of the order in which students took the two assessment forms — related
• Checklist tool use significantly increases Evaluation processes, an effect not observed in the ChatGPT and human-expert groups — related
• High-school students do not spontaneously use contextualization, sourcing, and corroboration heuristics when reading documents — related
• Trained students increased references to the more reliable source in justifications, specifically for the more reliable source — related