Develop a Reliability Checklist¶

Strategy · All strategies
Evidence · no claims cited

Description¶

Learners work together to generate a set of diagnostic questions for judging the reliability of online sources — e.g., Who is behind this site? What evidence do they cite? What do other sources say about it? — and then apply that checklist to real web pages. The strategy combines co-construction (which builds ownership and surfaces learners' naive criteria) with structured application (which turns criteria into habitual evaluation moves).

Design Implications¶

Co-constructing evaluation criteria converts source evaluation from an implicit expert skill into an explicit, reusable procedure, addressing the well-documented finding that students rarely attend to source features spontaneously Britt & Aglinskas showed students largely ignore source information unless prompted. [+S]. The checklist functions as a Scaffolding device: it externalizes the evaluation process so novices need not hold criteria in working memory while reading. However, the checklist must be taught alongside how to answer its questions — professional fact-checkers answer "who is behind this?" by leaving the page and reading laterally, not by scrutinizing the page's own "About" section Lateral reading outperforms on-page evaluation for judging source reliability. [+S].

Context¶

Requirements¶

• Authentic web sources (including deliberately unreliable ones) to evaluate
• Instructor modeling of how to answer each checklist question, ideally via Think-Aloud demonstration of an expert evaluating a live page
• Time for learners to apply the checklist to multiple, varied sources — a single application does not build the habit
• A mechanism for revising the checklist after application, so learners confront cases their criteria fail to handle

Constraints¶

• Checklists that can be answered from surface features alone (e.g., "Does the site look professional?") actively mislead learners; professional-looking misinformation passes such tests easily Surface cues are unreliable indicators of credibility. [-S]
• Learners often treat checklist completion as the goal rather than evaluation itself, producing perfunctory "yes" answers without genuine judgment Students evaluate sources superficially unless evaluation is embedded in a meaningful task. [~M]
• Static checklists decay as the information environment changes; criteria tied to specific platforms or formats need periodic renewal
• Co-construction without expert input risks enshrining naive criteria (e.g., ".org sites are trustworthy") that must later be unlearned

Implementation Variability¶

• Co-constructed vs. expert-provided: co-construction builds buy-in and metacognitive awareness; expert-provided checklists (e.g., based on fact-checkers' moves) are more accurate starting points. A hybrid — learners draft, then compare against professional fact-checkers' heuristics — captures both benefits
• Checklist + lateral reading: pair each checklist question with the action that answers it (e.g., "open a new tab and search the organization's name"), following the Cognitive Apprenticeship modeling–coaching–fading sequence
• Rubric vs. heuristic: shorter heuristic lists (3–5 questions) are more likely to transfer to independent searching than long rubrics

Target Learners¶

• Secondary and post-secondary students conducting research for the first time, who default to evaluating sources by surface features Novices rarely attend to source information unprompted. [+S]
• Learners with some web experience but no explicit evaluation instruction — the checklist names what they do intuitively and corrects what they do wrongly
• Less necessary for learners already trained in lateral reading, who have internalized evaluation moves Expert fact-checkers evaluate faster and more accurately than trained students using checklist-style on-page reading. [~M]

Target Learning Goals¶

• Source evaluation and information literacy: judging credibility, bias, and evidence quality
• Epistemic cognition: understanding that reliability is a property to be established, not assumed
• Transferable procedural skill: applying a diagnostic routine across unfamiliar sources and topics

Instructions¶

1. Present two contrasting sources on the same topic (one reliable, one not) and ask learners what distinguishes them; collect candidate criteria from the discussion
2. Model expert evaluation of a live page via Think-Aloud, making visible the moves (leaving the page, searching the author, checking citations) — not just the conclusions
3. Co-construct the checklist, pushing learners toward questions that require leaving the page ("What do other sources say about this organization?") over surface-feature questions
4. Have learners apply the checklist in pairs to new sources, then Class Discussion of disagreements — divergent reliability judgments are the productive moments
5. Revise the checklist based on cases it handled poorly, then fade it: learners evaluate new sources without the checklist and report which questions they still asked from memory

Related Strategies¶

• Lateral Reading Practice — the evaluation moves that answer checklist questions; the checklist without lateral reading degenerates into on-page inspection
• Contrasting Cases Analysis — the initial comparison of reliable vs. unreliable sources that launches checklist co-construction
• Think-Aloud Modeling — how the expert's evaluation process is made visible before learners apply the checklist

Examples¶

• Stanford History Education Group (SHEG) "Civic Online Reasoning" curriculum (https://cor.stanford.edu) — free lessons in which students learn fact-checkers' heuristics (including "investigate the source" and "find better coverage") and apply them to real social media content; grounded in the Wineburg & McGrew research program
• NewseumEd "E.S.C.A.P.E. Junk News" (https://newseumed.org) — a structured framework (Evidence, Source, Context, Audience, Purpose, Execution) learners apply to news artifacts
• A middle-school media literacy unit where students co-generate a class checklist, apply it to three websites on a controversial topic, and defend their reliability rankings in a structured discussion

Key Sources¶

• Wineburg, S., & McGrew, S. (2019). Lateral reading and the nature of expertise: Reading less and learning more when evaluating digital information. Teachers College Record, 121(11), 1–40. doi:10.1177/016146811912101102
• Britt, M. A., & Aglinskas, C. (2002). Improving students' ability to identify and use source information. Cognition and Instruction, 20(4), 485–522. doi:10.1207/s1532690xci2004_2
• Walraven, A., Brand-Gruwel, S., & Boshuizen, H. P. A. (2008). How students evaluate sources and sources of information when searching the World Wide Web for information. Computers & Education, 52(1), 234-246. doi:10.1016/j.compedu.2008.08.003
• Brand-Gruwel, S., Wopereis, I., & Walraven, A. (2009). A descriptive model of information problem solving while using internet. Computers & Education, 53(4), 1207–1217. doi:10.1016/j.compedu.2009.06.004