Learning Design Wiki — Agent Operating Guide¶

This is a persistent, LLM-maintained knowledge base for learning design. The wiki compiles design principles, instructional patterns, elements, strategies, theories, learner variables, and empirical claims into a structured, cross-linked reference.

You never write the wiki yourself. The LLM reads sources, ingests new content, cross-links pages, and keeps schemas consistent. You source materials and ask questions.

The wiki is a bundle in the Open Knowledge Format (OKF) v0.2: every content page is markdown with YAML frontmatter carrying type, title, description, status, generated, and (where applicable) sources; cross-links are plain bundle-relative markdown links, not Obsidian wikilinks; index.md and log.md are OKF's reserved directory-listing and change-log filenames.

Working alongside other sessions — read this before starting¶

This repo has one human maintainer and many Claude Code sessions — nine open at the time of writing, each in its own worktree. A session that has been idle for a day still holds its old plan and its old view of main, and resuming it will happily redo work that has already landed. Sessions cannot message each other (an idle one is disconnected), so this file is the coordination channel — it is loaded into every session's context automatically. Anything a future session must not undo belongs here.

Keep the number of live sessions small. Most of the duplication below came from parallel sessions, not from parallel people. One session per open task, closed when the task merges.

Branch from main, merge to main the same day, never stack. A wiki-wide fix is a wiki-wide diff: a typical task here changes 200+ files. Two branches living longer than a day will collide on hundreds of them. This is not hypothetical — on 2026-08-31 three branches were open at once, each changing ~200 of the same files, arranged in a stack (normalize-slugs → fix-no-h1-pages → research-scraper-test-setup) whose base kept moving. Two sessions independently fixed the same 13 corrupted pages, independently wrote a link-repair tool, and independently shipped the same [^)\s]+ parenthesis bug in it. One merge cost 41 conflict resolutions. Do not open a PR whose base is another feature branch.

Before starting any wiki-wide pass, check whether it is already done. Run python3 scripts/lint.py and python3 scripts/wiki_health_check.py --skip-doi first, and git log --oneline -20 origin/main to see what landed recently. A count of zero means the work is done, not that the check is broken.

Things that are settled — do not re-litigate or revert:

Never assert a DOI that has not been resolved against Crossref, or, after a Crossref 404, against DataCite, with the same title check (added 2026-09-26). A DOI that resolves to the wrong paper is worse than none, because it reads as verified. 10.1007/978-1-4684-7562-3_3 ("Model of Causality in Social Learning Theory") was auto-applied to 69 pages as Bandura (1977); it took a manual audit to catch. cc.titles_align() guards the containment case that let it through, and enrich.verify_page_citations() gates every newly written page.
classify_doi's error status is not wrong_paper. A failed lookup means Crossref was unreachable. Stripping on error deletes good DOIs from every page touched during an outage.
No content page is status: draft any more (maintainer, 2026-10-15: "we've moved past that stage"). Every page was promoted to review, and ingest, settling and enrichment now write review. draft is kept only for a genuine skeleton that a CSV import or enrich.py stub has not yet filled. review means model-written and not checked by a person; stable still needs a verified: entry. Lint's DOI check skips drafts, so with no drafts left it applies to every claim.
Links to filenames containing parentheses need the <...> form. A bare destination closes at the first ). scripts/fix_links.py --apply repairs them; lint's link_needs_angle_brackets catches new ones.
consent.basis: licence must keep its five conditions, and condition 4 above all. A licence basis satisfies the research layer's AI-processing check in place of a participant disclosure, which is correct for secondary data — but it is the only value in that schema that relaxes a check, so it is the only one that can be abused. It requires a named licence with its grantor, a non-empty secondary_use_caveats, ai_processing_disclosed: not-applicable, data.identifiability.collected of deidentified or aggregate, and (added 2026-09-15) sources_uniformly_governed: true. The fourth is what stops it becoming a one-line escape hatch: it is why lazuli-platform-telemetry cannot reach the path however its consent block is written. Simplifying these conditions away would silently re-open the check the layer exists to enforce. The fifth stops one licensed corpus buying the relaxation for every other source in a protocol that governs several — declaring sources_uniformly_governed: false is honest and costs the relaxation, which is the intended outcome, and the fix for a protocol needing both is to split it. Reasoning in research/SCHEMA.md ("Secondary data" and "One value, several instruments"), mutation list — now six, all of which must break the benchmark release — in the fixture header at research/protocols/example-open-corpus-secondary-use/1.0.0.yaml.
scripts/find_title_duplicates.py lists only pairs no judge has decided (2026-10-20). A pair two models judged different (eval/duplicates/<kind>.ndjson, eval/claim-merges/decisions.ndjson, both committed) is not listed again; --include-judged shows them. Every batch judges its new pairs, so the count stays near zero. It is reported, not lint-failing, on purpose.
Embargoed research never enters this repository, on any branch. learning-wiki is public: a pushed branch or an opened PR is visible to anyone, merged or not, so "keep it in an unmerged PR" does not keep anything private. Embargoed work lives in the private Learning-engineering-research repo until the embargo lifts, and only then is it written here. The private repo stamps embargoed files with the marker LDA-EMBARGO. scripts/check_embargo.py refuses to let it out, before the push (CI runs too late, since the branch is already public by then): a PreToolUse hook in .claude/settings.json blocks git push and the GitHub MCP write tools in every Claude Code session here. git config core.hooksPath scripts/hooks does the same for a person pushing by hand, and lint.py --type embargo is the CI backstop. A marker catches copying, not prose retyped from memory, so an agent with the private repo attached must still treat anything it read there as embargoed unless told otherwise.
No LDA research is published in this wiki yet, and none may be drafted here. research/ holds synthetic scaffolding only. The one real governance file, lazuli-platform-telemetry (which forbids publication), moved on 2026-09-24 to the private repo's transplant tree, Learning-engineering-research:research/wiki/research/ protocols/lazuli-platform-telemetry/1.0.0.yaml, byte-for-byte; the mentions of it below still describe it correctly, but it is not in this repo. A finding arrives only once it is public, by the route in research/PUBLISHING.md: a protocol permitting publication, a plan that predates the release, and a DOI reserved before the release file is frozen (scripts/mint_release_doi.py --reserve). Only a human: actor may --publish, which is irreversible. release.doi is refused on any non-research release, because a DOI is exactly the badge that makes a simulation read as a finding.
The repo root is public-facing: nothing but the wiki's own files lives there. On 2026-09-25 it carried 22 run logs and one-off reports (glm-*.log, doi-report-*.md, health-final.md, health-truly-final.md), the first thing a visitor saw on GitHub. They were removed (git history keeps them) and .gitignore now refuses them. Write run output to eval/runs/, which is ignored; never commit a log or a scratch report at the root.
A pattern is a reusable, research-based design, general across domains and contexts (maintainer, 2026-09-30). A course for a particular setting, such as a self-paced mobile language course or a clinical onboarding programme, is a design that uses the patterns whose evidence fits its learning goals and learners. It is never a pattern page. Context (mobile, no teacher, 10 minutes a day) is a requirement or constraint a pattern's page speaks to, and a design scenario is at most an entry under that pattern's ## Examples. Do not create context- or domain-specific patterns, and do not report "no pattern for this course" as a gap. Some patterns/ pages were designs in this sense (the Duolingo pages, kindergarten-balance-scale-equal-sign-lesson-sequence, tutor-led-strategies-adult-language-lessons, elderly-refugee-programs-beyond-language-learning, among others, now in designs/). Designs live in designs/ (maintainer, 2026-10-02), a kind of their own, with id: like the other identified kinds: the triaged ones move there from patterns/, and Lazuli-generated design descriptions are added there. A design page says which patterns it uses and for what (## Patterns Used); the evidence stays on the patterns and claims it links.
Products and programmes live in products/, and education research methods in research-methods/ (maintainer, 2026-10-10). A product is a named thing others adopt or use: software or a platform, a curriculum or programme package, an assessment or measurement instrument, a dataset, a branded intervention or initiative. One programme's own frameworks, indicators, tools and theories of change are listed on its page under ## Components, not written as theories or elements. A design is one setting's design (a course, a lesson sequence, one site's implementation); a branded programme many sites adopt is a product. A research method is a method for studying learning or evaluating education that is specific to education or especially useful there (cluster-randomised trials of schools, evidence standards, knowledge tracing, growth and value-added models). General social-science methods are not pages (maintainer: "regular old social science research methods are not that useful in a database like this"). methods/ stays the design methods. Design-based research is a design process (processes/design-based-research), not a research method (maintainer, 2026-10-20). research-methods/ is secondary and may eventually go away, so where a page could sit in either, it belongs with the design side. A review's or meta-analysis's set of included studies is not a product ("corpus of 36 studies", "meta-analytic dataset"): its findings are claims. A named dataset others reuse is. lint.py's misfiled_kind fails on either kind of misfiling, and settle_candidates.py refuses both (2026-10-21).

When you finish something wiki-wide, add a dated entry at the top of history/agent-log.md, newest first, as every session before you did; that file is how the next session finds out what happened and why. If the work changes a standing rule, change it below too. This file holds the rules; the history holds the reasons and the numbers (moved there 2026-10-09, when this file had reached 272 KB).

Standing rules from the history (read the dated entry before changing one)¶

The site and the generated files (2026-10-09):

The docs site has two tiers. mkdocs renders principles, patterns, designs, processes, methods, learner variables and the elements and theories marked canonical: true; scripts/build_longtail.py renders claims, strategies and the other elements and theories as light pages at the same URLs, with letter-paged listings. Pagefind indexes both. scripts/build_site.sh is the one build, used by all three workflows; docs_hooks/two_tier.py says which page is in which tier. PR previews run only for PRs that change the site build (or carry the preview label).
Pagefind is pinned at 1.5.2 with its binary (pagefind-bin-extended): 1.3.0 silently dropped results for any word at an index-chunk boundary. The search page lists Main pages (canonical, then other curated pages, tier filter, re-ranked by title and alias coverage) above the full results, turns off Pagefind's long-page penalty, and every page has a header search box.
Deploys replace the published tree (keep_files: false), carrying open PR previews over, so renamed and deleted pages do not accumulate. At about 21 KB a page (measured 2026-10-12), the 1 GB Pages limit is near 48,000 pages; past that the long tail has to move off GitHub Pages.
log.md holds the current month; okf_lib.rotate_log() moves earlier months to log/YYYY-MM.md. The log is not on the site.
wiki-index.json and reverse-index.json are written one record per line (okf_lib.dump_json_records). Change their shape only additively. No copy of the index lives anywhere else (checked 2026-10-20: not in the Lazuli app, not in learning-design-spec, whose docs describe a platform copy that was never committed). Its readers read this repo directly: Learning-engineering-research's wiki_agent.py reads pages from a local checkout, and the spec's check-wiki-links.py checks page URLs. So there is nothing to keep in sync.
canonical: true marks the pages the candidate ledger settles against (settle_candidates.py --mark-canonical), and every learner-variable page. Search ranks them first when a query names them.
situation-index.json is generated from every ## Fitting the design to a situation table (build_situation_index.py, run by build_indexes.py, --check in CI). It feeds the MCP situations tool and the site's "Design for a Situation" pages. Write a row's situation with the evidence-dimensions.json tags (`novice`, `online-self-paced`, `procedure`) where one fits: a row named only in words is found by a few conservative patterns or not at all.
python3 scripts/eval_search.py scores search against eval/search/designer-queries.json; run it before and after a ranking change, and add the query a change is meant to fix.

Batches (scripts/run_scrape_batch.py):

Curated lists come first (maintainer, 2026-10-07): the Learning Agency hub queue, then the AIED knowledge base (scripts/eval/source_lists.py writes both to eval/runs/queues/, ignored; the hub has no licence, so none of its data is committed). Run --queue <files> --take N --pmc 0 --eric 0 --arxiv 0. Unfetchable entries go to eval/runs/queues/fetch-failed.ndjson. Campbell reviews are fetched by their PMC copy (--campbell), DSpace repositories through their REST API.
Batches write no principle, pattern, element or theory page. Ingest puts each in eval/candidates/candidates.ndjson, and settle_candidates.py --apply (run in every batch) attaches it to a canonical page, joins it to another candidate, writes a design page, writes it to its product's page (product, grouped by name: a programme's tools and frameworks become components of one page) or a research-method page, or drops it. The old artifact outcome is no longer given, and its candidates are asked again. Promotion to a new canonical page is an agent's job, checked by scripts/check_design_page.py. Do not re-enable --direct-pages. A cluster promotes on two independent sources: different articles with no author in common and not published by the same organisation (maintainer, 2026-10-10: one author or organisation across years is a research agenda, not confirmation; settle_candidates.independent), or on one synthesis at q3 or above.
Pages from before the two kinds were re-filed once (scripts/sweep_kinds.py, 2026-10-10): 1,323 to products, 222 to research methods, each move confirmed by a second model. Its 390 plain folds into canonical pages are in eval/candidates/sweep-kinds.ndjson and were not applied (too many loose matches); apply them only after review, never with a bare --from ... --apply.
A claim needs a DOI, an evaluation report, or a design page that leans on it (maintainer, 2026-10-15: "quality over quantity"). A claim whose evidence has no DOI after a Crossref and DataCite search is removed, unless it rests on a report from a named evaluation organisation (WWC/IES/REL, Mathematica, MDRC, AIR, RAND, Abt, SRI, WestEd, Westat, Child Trends, the UChicago Consortium, NWEA, LPI, Urban Institute, CRESST, CREDO; maintainer: "evaluation reports are useful") or a principle, pattern or 10+ design pages cite it. A kept claim of the last kind carries a > **Evidence quality** · line saying what it rests on. New sources come from syntheses and from chase_citations.py, not from ERIC or arXiv topic search.
A claim is kept when a design decision could change because of it (maintainer, 2026-10-13). That covers evidence about a design choice, the reliability of a component a design relies on (knowledge-tracing accuracy for an adaptive course, essay-scoring or AI-detector accuracy), and context that informs a design (needs, learner populations, baselines for reading an effect size). 1,959 context claims removed on a misreading of the maintainer were restored the same day; claims are re-sorted on this test (eval/claim-sort/, sort_claims.py --all). Research-practice claims are removed where both models agree (maintainer, 2026-10-16: "we don't need research methods studies that don't directly impact design decisions"); a disputed sort stays. Local, dated context still goes to the maintainer as removal candidates. scripts/remove_pages.py does a removal.
Claims about a learner characteristic link into its learner-variables/ page and back (scripts/link_learner_variables.py, --new in every batch): learning-design-spec's learner dimensions reach the research only through those pages. The model names the role (predictor, moderator, outcome) and a marker capped at the claim's evidence; the claim gets a ## Learner Variables section.
Group categories are never written as causes (maintainer, 2026-10-16: "race is not a contributor to learning outcomes but does correlate with other factors"). On race-and-ethnicity, gender and socioeconomic-status, a claim is linked as "outcomes differ by group: a gap reflecting circumstances and treatment, not a cause" (link_learner_variables.ROLE_TEXT_BY_VARIABLE), and each page opens with a How to read this page statement saying so. Learning-variable headers count "show a difference / mixed / no difference", never "for / against" (evidence_rollup.POLARITY_LABELS). learning-style-preferences opens with its bottom line: matching instruction to style is not supported; single studies listed there are not tests of matching.
The coded evidence cells are committed (eval/runs/evidence-axes/cells/ and cells-contrasts.json; maintainer, 2026-10-13), the one folder under eval/runs/ that is: every evidence map and report reads them, and the ignored store vanished with each container. code_evidence_axes.py refuses structurally broken replies, so a bad cell is retried, not stored. The store is sharded by the first two characters of the claim slug (cells/sp.ndjson; maintainer, 2026-10-24: "easiest to search and maintain as it grows"), read and written only through scripts/evidence_cells.py. Do not gzip it or rejoin it into one file: the single file reached 53 MB against GitHub's 100 MB limit, and a compressed file is a whole new blob in history on every batch.
A fresh container needs pip install -r requirements-eval.txt (without pypdf most PDFs fail).
After a container restart the proxy port changes; a process started before it keeps the old HTTPS_PROXY and every request fails. Compare /proc/<pid>/environ with the current value, kill and --resume.
Every batch cleans up after itself (2026-10-20, after an overnight wiki-wide cleanup the maintainer did not want to repeat). run_scrape_batch.py now, for the batch's own pages: links its new claims into concept pages (link_pages.py --orphan-claims --only, verified); merges claim pairs two models judge the same finding (judge_claim_merges.py --judge --titles, then --apply); judges new same-folder title pairs (judge_duplicates.py, only pairs no judge has decided); links inline author-year citations (link_inline_citations.py); re-renders the study tables; and judges every claim three or more design pages cite (check_load_bearing.py --top 5000, only changed entries are paid for). What a model cannot settle stops the run for a person or agent: a load-bearing failure, a new citation problem, an unmarked claim citation, an i1–i3 with no printed statistic, an unjudged cross-folder collision. Each of those health counts was taken to zero on 2026-10-20 and must stay there.
Do not commit hand-written pages while a batch is running, and check a fresh-container batch's claim diff before committing: a step whose correctness depends on an ignored cache is a bug.
CURRENT is prompt v136. No provider pin; leave OPENROUTER_MAX_COMPLETION_PRICE and OPENROUTER_QUANTIZATIONS unset. :batch models are refused by the account's zero-data-retention setting. Costs recorded before 2026-09-28 are list-price estimates.
Topic search (--pmc, --eric, --topics-file) adds volume, not decisions; prefer named syntheses and the curated lists.

Evidence on claim pages:

Every entry carries q (design tier), i (impact), n, and a kind with its rigour (qualitative · r3), judged within the kind (eval/kind-rigor/RUBRIC.md) and never from the wiki's own paraphrase. code_kind_rigour.py --new runs in every batch.
An i1–i3 must rest on a printed effect size for that finding; otherwise i? (recode_impact.py). i? is never 0.
An evidence entry opens with its citation line; the tools take its first link as the resource.
Count distinct studies, never claims; no verdicts or averaged i in roll-ups.
A field added to the codes line must be left out of every digest that keys a verdict (check_load_bearing.units()).
Rigour is never re-judged in a merge or cleanup from the entry text.
The load-bearing check (check_load_bearing.py, runbook eval/load-bearing/README.md) runs in every batch; read a judge's failure before acting on it, and --dismiss a checked one.
Markers are capped by the claim's recorded evidence (strength_cap).

Principle and pattern pages (conditional-model format, principle-pattern-authoring.md; waves 1–7 in eval/page-triage/):

Every principle and pattern page is brought up in one loop (maintainer, 2026-10-14; they are what a course design names). scripts/upgrade_worklist.py gives each page its step: search (a canonical page on two studies or fewer: unlinked wiki claims via link_pages.py --pages ... --claims-only --verify, then find_page_syntheses.py, which names published work and keeps only what Crossref confirms, and a batch on its queue), fold (still thin after the search: into the parent choose_fold_parent.py finds, two models agreeing or a third settling, with merge_pages.py; maintainer: placeholders "that haven't really held up to the research"), convert (three studies or more: the conditional format below), fragment (not canonical: folds into its canonical page) and reread (converted, with more than ten unread further-evidence bullets). Do not convert a thin page instead of folding it.
Where no claim tests the page's relationship, keep a labelled default design with doses and a progression across sessions; a rewrite that leaves only caveats loses.
## Fitting the design to a situation rows: take the brief's own categories, routine and privacy terms as given; when a page loses a test, look first for the missing setting row.
A page stating an absence of claims says it is an absence in the wiki, not in the literature.
Keep every claim the old page cited; keep the old body in a  block. Links inside HTML comments count for nothing.
Folds and merges: merge_pages.py (across kinds with kind/slug, no alias) and merge_claims.py (keeps a differing write-up of a shared study in a  comment). After merging claims run dedupe_claim_bullets.py --apply: a page that cited both lists the kept claim twice in a section, under the two old titles (judge_claim_merges.py runs it itself). Which page survives is the maintainer's call when it is not obvious.

Citations:

Crossref (and DataCite after a Crossref 404) is reachable from this sandbox; check once per task. Identity is checked on first author and year as well as title (citation_identity.identity_mismatch); a foreign-language registry title passes only with author, year, journal and a coordinate (translated_record).
Never fill a DOI by majority; resolve_doi_conflicts.py --apply needs --allow-cluster-rewrite. Run verify_citation_edits.py after any citation tool writes.
Never branch a run off whatever branch is checked out: git checkout main && git pull first.

Housekeeping:

Branch lists in prose go stale; run the command. Every remote ref against main: for b in $(git for-each-ref --format='%(refname:short)' refs/remotes/origin); do echo "$(git rev-list --count origin/main..$b) $b"; done | sort -rn
If something merged and the site does not show it, check the Deploy Docs run first.
Third-party Actions are pinned to commit SHAs; resolve a new tag with git ls-remote.
python3 scripts/wiki_health_check.py is the general health report; scripts/health_evidence.py its evidence section.

Known open work¶

Kept in history/agent-log.md with the state notes it refers to. The parts still open as of 2026-10-09: the sweep of older pages onto current conventions (priority_worklist.py, check_evidence_markers.py), the citation backlogs (check_citations.py and its --collisions, --metadata, --titles, --variants modes; citation_worklist.py for the book backlog), and the maintainer decisions listed in the 2026-10-09 history entry; the kinds and the independence rule were settled 2026-10-10.

Research observations — the structured record beside a claim¶

A claim page argues something. observations/<study-key>.yaml records the configuration that argument rests on, so a study can eventually be represented as

intervention/configuration + population + context + comparison + measurement + time
    → observed outcome/effect

and so that d = .78 stays one measurement of one configuration rather than becoming a weight between two concepts. Two studies of "retrieval practice" with different learners, implementations, outcomes or horizons are two records and must stay two records.

observations/SCHEMA.md is the field reference; scripts/observation_lib.py carries the rationale. What belongs here:

The schema is at version 2, and the change was forced by evidence rather than foreseen. v1 had two levels — a study whose population/context/intervention were assumed SINGULAR, and observations under it. That middle assumption is a primary-study assumption, and it broke in nine places the moment a meta-analysis and a three-arm trial were put through it: 23 studies and 3371 participants had one integer slot between them; k differed per pooled estimate (9 for knowledge, 2 for surgical skills, from a corpus of 23) so it could not live on the study; three arms had one intervention: field; and a comprehension ANOVA on 62 of 67 randomised had no home for its own analysed n. So there are three levels now — source, evidence base, observations — and the middle one carries arms.

The governing rule, which explains every placement in the schema: put information at the lowest level at which it actually varies, and preserve observations at the grain at which the source actually reports them. Total corpus N varies per study; analysed N per result; k per pooled estimate; arms and comparisons per contrast. And a pooled estimate over heterogeneous comparators stays ONE observation, because that is the grain the source reports — the heterogeneity is recorded structurally, never normalised into effects nobody measured.

Arms + comparisons is the abstraction that makes one schema fit both shapes. A three-arm trial's "unenhanced vs baseline" and a meta-analysis's "spaced online vs massed online" are the same object: a contrast between two named configurations. The synthesis pools it across studies; the trial measures it once. Nothing about a meta-analysis needed a parallel set of fields — it needed the middle layer to stop assuming one group of people. An arm carries no role. The first draft gave it one and it was removed on evidence: nothing read it, and it is not a property of an arm — in the three-arm fixture unenhanced-elaboration is declared intervention and is the reference side of enhanced-vs-unenhanced, so the role varies by comparison, which is where index_arm/reference_arm already say it. Nor does an arm need a type to mark a synthesis: the source already says design.family: meta-analysis and the arm's own size: {value: 17, unit: studies} says it is studies rather than people. What replaced role is the invariant it was standing in for without enforcing — every arm must be named by at least one comparison, because an arm nothing is contrasted with is unreachable from any observation. A single-group study has one arm; a survey has arms: [].

Either side of a comparison may name more than one arm, which is the whole representation of a heterogeneous comparator. Martinengo et al. pool three studies against two different controls and report one estimate, so it stays one observation; the compiler emits reference_is_heterogeneous: true beside the resolved arms. Per-study contribution counts are deliberately not recorded — the source reports them, but no use case needs that grain yet, and a field nobody reads goes stale.

A cluster design needed four fields and no new level, which is the strongest evidence so far that the abstraction holds. evidence_base.allocation records the unit RANDOMISED where it differs from the unit analysed — 20 classes against 370 students, with every group contrast carrying df = 16 rather than 369, so reading size as the effective sample overstates precision by the design effect. result.clustering holds the ICCs and design effects, as a list with a required level, and deliberately NOT folded into heterogeneity: I² is between-study variation of effects, an ICC is within-study correlation among observations sharing a cluster, and pooling the two would be pooling nonsense. result.power records reported power with kind required, because a post-hoc 0.44 is what separates "no effect" from "no detection". And comparisons[].contrast carries weights where contrasts are not pairwise — Helmert H1 sets control against the mean of three arms, which the arm lists already handled.

A network meta-analysis broke the one rule that had held through every other shape: comparison_ref was required on every observation. A SUCRA ranks one model within a network and rests on no contrast, so an observation now names exactly one of comparison_ref or arm_ref. With it came result.interval_type (a Bayesian credible interval is not a confidence interval, and unlabelled bounds say nothing about which), comparisons[].evidence_route (in a network an estimate may rest on no head-to-head study at all), and result.publication_bias — an assessment RUN AND FOUND, as distinct from attempted_but_precluded, which is one that could not be run. And the schema caught itself: result.k was required on every pooled effect until an NMA put per-contrast counts in a figure, leaving only "invent a k" or "break the schema". observability.pooled_k restores the absent-versus-unreported distinction to the rule that had forgotten it.

A single-case design needed the SUBJECT to have a name. In a group design the people are a sample and only their count matters; in a multiple baseline each participant is a separate replication, the findings differ by person, and "three of the four maintained" is unreadable without knowing which one did not — it was Serena. evidence_base.subjects and observations[].subject_ref, symmetric with arms, absent for every group design. With it: tau_u (single-case designs use non-overlap statistics, and its siblings are not added until something needs one) and result.estimate_range with a required over — Spencer & Kirby report a Tau-U range of 0.64–1.06 across four participants and no pooled estimate anywhere, so the alternatives were to invent a midpoint or drop the effect. The phases are the arms: baseline, intervention-with-icons and icons-removed are three named configurations and a phase change is a within-subject contrast.

All three predicted-hard shapes now encode, and arms + comparisons never moved. Nine studies, 36 observations: four second-language primary studies, a meta-analysis, a three-arm trial, a four-arm cluster-randomised trial, a Bayesian network meta-analysis and a multiple-baseline single-case design. Every addition across the three sat at the level the thing actually varies at, which is the governing rule doing its job rather than being quoted at one.

Every count carries its unit, and the corpus proved that necessary before either fixture was written. Its evidence entries already carry n=18, n=30 studies, n=66 articles, n=N/A and n=large (aggregated) — one field doing four jobs, so nothing can read it. A count here is {value, unit} with unit required.

There are four epistemic states, not two. Absent (nobody looked); unreported (somebody read the source and it does not say); attempted and precluded (the authors tried the analysis and could not finish it, with the reason); and measured-as-null. Martinengo et al. attempted subgroup analyses and publication-bias assessment and were precluded by the number of studies and their heterogeneity — unreported would lose the reason and absent would lose the attempt, so study.synthesis.attempted_but_precluded holds it.

appears_in is optional in v2. Requiring it made the evidence layer depend on the argument layer: a structured record of a real study is valid evidence whether or not anyone has yet written a claim that appeals to it. The ratchet stays — a listed claim and anchor must still resolve — and check_observations.py --summary counts records nothing cites rather than letting orphans go unnoticed.

The record is NOT in the claim's frontmatter, and that was decided by reading the code. sources[] is derived: sync_evidence_codes.py --apply rebuilds the whole block from the body's ## Evidence through dump_frontmatter, which emits exactly id/resource/title/author/q/i/n — so anything else nested there is destroyed on the next run of a script that is already in run_scrape_batch.py's unattended chain. And check_source_entry_keys holds a closed key set at four-space indent precisely so a rewrite landing on a YAML key is caught; a new key there is indistinguishable from that damage. Third reason, from the data: a study is cited by many claims, so a per-claim copy is the drift shape this repo has already lost weeks to on DOIs.

No existing claim page changed. The join runs from the record: appears_in names the claim slug and the ### Author Year anchor, and lint.py --type observations fails if either stops resolving. A rename breaks loudly instead of orphaning the record silently.

Three distinctions the schema exists to hold, and none of them may be collapsed:

Absent ≠ unreported ≠ zero. An omitted field means "not established"; observability.<field>: unreported means "somebody read the source and it does not say". A consumer reading a missing effect size as zero drags every pooled estimate toward nothing. Same discipline as crossref_reachable: false vs flagged, "doi": null vs an absent doi, and classify_doi's error vs wrong_paper.
A capability is not a valued outcome. outcome.role records which a dependent variable appears to be and outcome.role_basis records whether the source said so or you inferred it; outcome.source_language keeps the author's wording regardless. outcome.valued_by is populated only where the source identifies who values an outcome — a researcher measuring graduation does not establish that anybody values it.
A research observation is not a design hypothesis. A logic model says "Goal A is intended to contribute to Outcome B"; a record here says "Study S observed relationship R between configuration A and Outcome B under population/context C". There is deliberately no field in which to write that one becomes the other, and every compiled record carries kind: research_observation and provenance.source_type so a published finding stays distinguishable from runtime learner data, a synthetic simulation, an LLM-proposed relationship, or a later platform experiment.

Never infer a stronger design than the source states. quasi-experimental is what "the sample was divided into two groups" means; an article calling for randomised trials as future work has not run one. moderators.reported requires its own source_quote — a plausible explanation goes in candidate, and promoting one is the same act as inventing a DOI.

Never convert a metric during ingestion. A partial η² stays a partial η²; turning β = 17.17 words into a d needs a pooled SD, and inventing one to make records comparable is how a store fills with confident untraceable numbers. compile_observations.py therefore does no pooling, no averaging and no conversion — comparability is a separate analytical step run explicitly downstream. A prediction interval is not a confidence interval and must never be derived from one: the CI is about the mean effect, the PI about the next study, so result.prediction_interval.source is required and a computed one can never be read as a reported one.

python3 scripts/check_observations.py                 # validate; exit 1 on any problem
python3 scripts/check_observations.py --summary       # what is in the store, by shape
python3 scripts/check_observations.py --stub <claim>  # skeleton for a claim, TODOs and all
python3 scripts/compile_observations.py               # research_observation NDJSON
python3 scripts/compile_observations.py --explain <study-key>/<obs-id>

--explain is the acceptance test run as a command: what exactly was done, to whom, compared with what, in what context, measured how, at what time, with what result and uncertainty, and what important information was not reported — answered from the record alone, with no narrative prose read.

Migration is deliberately not a batch job, and nothing has been migrated. Six studies are recorded as fixtures — four second-language primary studies, one meta-analysis (martinengo-2024) and one three-arm randomised experiment (jemr-lexical-elaboration-2026), 21 observations in all. 424 claim pages have no record and are not broken by that, because an absent record is a legal state and check_observations only validates the files that exist. It starts and stays green, so the property this file leans on — that a count of zero means the work is done rather than the check being broken — survives.

The four v1 fixtures were converted to v2 and the conversion was verified, not eyeballed: every page's observations, study and appears_in block parses byte-identical before and after, and population/context are preserved with only n moved into evidence_base.size. That is the same discipline the 115-page frontmatter repair used — "the parsed value is unchanged except where I intended it to change" is checkable, and reading a diff at that size is not.

The route for the rest, when the structure has proved itself:

--stub <claim-slug> fills in what is genuinely mechanical (citation, DOI, the claim/anchor join, an ASCII-folded study key) and marks everything else TODO. The validator refuses the file until the TODOs are gone, so the cheap half is automated and the half that needs the article is blocked rather than guessed.
Rank by reuse, not by slug order. A study cited by 30 claims repays a record 30 times; citation_worklist.py already ranks by exactly that and is the right reading end.
Prefer the source over the page. Two of the four fixtures were built from publisher full text, one from an abstract, one only from the wiki's own evidence entry — and provenance.extraction_method says which, per file, because the three are not equally good. Building a record from a paraphrase of a paraphrase is how the fabrications this file catalogues got in.
Automated extraction is built end to end, and not switched on. prompt_versions/v129.txt asks for one v2-shaped study_record per article (arms, comparisons, observations, each observation linked to its claim's evidence entry with a bearing), plus arms[].delivery and design.preregistration. ingest_extractions.build_study_record turns it into observations/<key>.yaml only if observation_lib.validate_record passes against the live claim index. It never overwrites an existing record and never defaults a bearing, and it sets an anchor only on a claim page the same run wrote. The extraction validator reports a bad record as warnings, so a flawed record never costs an article its claims but does show in the tuning signal. v127 (a v1-shaped block that nothing ingested) is superseded. CURRENT pointed at v99 until 2026-09-27 and is now v135. Moving it is the eval ratchet's call, not a side effect: tune from v128 (the null-impact rule only) or v129 (plus the record), and let optimize decide.

One fixture corrected the claim page it came from, which is the point of doing this. claims/game-based-practice-outperforms-traditional-l2-vocabulary-instruction.md renders Frolli et al. as finding Italian articles "disproportionately hard for this group". The paper reports articles as an instructional accommodation — one example ("for instance") of complex topics taught with occasional Russian, because articles are absent from Russian and Ukrainian grammar — and measured no article-specific outcome at all. The observation file records it as an implementation note and a candidate moderator, never a reported one. Structuring a study is a re-read of it, and a re-read finds things.

The research layer — where knowledge came from¶

research/ records what happened and how it can be interrogated; the wiki's kinds record what we currently believe. Full field reference in research/SCHEMA.md; rationale in scripts/research_lib.py.

Protocol ──> Study ──> Research Release ──> Evidence ──> Claim
                                               ^
                                      Review ──┴── Issue

There is no evidence/ folder, and that is the main decision. observations/ already is the evidence layer — its study: block is "the SOURCE", its observations are the individual results with their arms, comparisons, samples and observability. A second Evidence object would hold the same fields about the same results, which is the drift shape this repo has lost weeks to on DOIs. So a release names its findings (evidence: [{ref: <study-key>/<observation-id>}]) and each record names the release back (study.release: {ref, version}). Both halves are required to agree: a half-written edge reads as a working link from whichever side you arrive on, and only a reader who checks the other side ever finds out.

Three additive fields in observations/, and one that became required. study.release (replacing citation when the source is ours — a release already carries title, authors, version and date), observations[].analysis_ref (which declared analysis produced this result), and appears_in[].bearing — supports / contradicts / qualifies, required and never defaulted, because the direction is a property of the edge: one result supports one claim and qualifies another. The four existing records were updated; the schema stays at version 2, so an older file fails loudly on the missing bearing rather than being read wrongly.

appears_in[].anchor is now optional, and that is the separation between the two layers. A record may name the proposition it bears on before anyone has written it into that claim's argument. Promoting evidence into a claim's ## Evidence section is an editorial act and stays one — do not automate epistemic promotion. An anchor, when given, is still ratcheted: a rename fails lint rather than orphaning the record silently.

Immutability is structural, not a promise. A version is a file — research/releases/<id>/<version>.yaml — so publishing 1.1.0 writes a new file and cannot touch 1.0.0. Which version is current is derived (highest semver present) and never stated, because stating it would mean editing an older file to say it is no longer current. The cost is real: 1.1.0 repeats almost everything from 1.0.0, because a version has to stand alone to be citable on its own.

A protocol is a configuration file, not a checkbox — and it records governance, it does not constitute it. external_review.status is a five-value vocabulary (approved / exempt / not-required / not-determined / pending) because "an authority determined it exempt" and "we reasoned none applies" are different claims by different parties. It is deliberately not spelled yes/no: YAML 1.1 reads a bare yes as the boolean True, which broke the first draft of the fixture.

Three enforcement checks are implemented, narrowly, to prove the arrow machine-readable protocol → automated checking is real: a dataset may not be more identifiable than its protocol permits at its access class; an analysis declaring ai_processing: true requires consent.ai_processing_disclosed: true (unspecified fails as well as false — silence is not permission); and a release requires permitted_uses['research-publication'] == permitted on its protocol, because a release IS a publication. All three were verified by mutating the fixture until they fired.

consent.permitted_uses is a mapping of closed classes, not a list of sentences, and the first real protocol forced that on day one: platform telemetry collected under terms of service, with no research consent step, may legitimately validate a pipeline and may not be published as research. As prose that line is unenforceable; as classes it is a release being refused. An absent class means unspecified, so absence is safe and growing the vocabulary cannot invalidate a frozen protocol file. secondary_research_use was a separate flag and is now refused by name — it is permitted_uses: {secondary-research-by-others: ...} said twice.

Review is a structured contribution, not a comment, and never evidence. assertion and rationale are required and the target must resolve, so a bare "interesting study" or an insult with no assertion never becomes an object — which is what makes open review expensive to spam without gating it. No institution, rank or degree is required, requested, or able to gate whether a contribution exists. What differs between contributions is author.basis — what grounds this contribution, a property of the contribution and not of the person.

The schema refuses positive, approved, helpful_votes, votes, score, reputation, credentials and bearing by name, each with its reason, enforced rather than requested. A review carrying bearing: is rejected because supports/contradicts/qualifies already exists on the evidence→claim edge; a review may introduce evidence, it is not evidence. assessment: records checkable properties of form (specific, reasoned, evidence supplied, independently reproduced, duplicate-of) — form is what automation may judge, and truth is not.

Moderation is conduct, never correctness. moderation: is a list (a history that can be overwritten is not a record), and the reason codes are a closed set of conduct categories with no code meaning "wrong" or "disagrees with the authors" — so burying a critique on the merits requires inventing a code the schema does not have, and fails. withheld changes prominence, never existence.

Issue is the sixth object, and it earns its place. Without it, eighteen people independently noticing one defect produce eighteen unlinked reviews: nothing can tell they are one problem, prominence has to be computed from volume, and the first person to say it is indistinguishable from the seventeenth. raised_by is ordered by first appearance and append-only. A resolved issue keeps every review that raised it and every review that disputed it; resolution.decided_by is required because a resolution is a position held by somebody, not a verdict issued by a platform.

A contributor's history must stay derivable and must never be stored. "Eleven concerns later confirmed, two not supported" is a traversal of author.id → reviews → issues → resolution. That is why reputation is a refused field.

The first real protocol exists, and its central statement is a refusal. (It now lives in the private Learning-engineering-research repo, not here; see the settled items above.) lazuli-platform-telemetry/1.0.0.yaml records the governance of Lazuli platform learning telemetry as it stands: learners accepted terms of service, nobody asked them to consent to research use, so the data may build and validate the observation pipeline and may not be published as research — permitted_uses: {research-publication: prohibited}. Naming it from a release fails validation, verified. Every field was established by reading learning-engine-ai-frontend (schema.ts, learning-graph/compile/runtime.py, report/gate.py, sdk/capture-consent.ts) and every field nobody has checked says unspecified or NOT ESTABLISHED — retention, the storage controls, the jurisdiction, the deidentification key holder. Do not fill those in by assumption; each is a question for a person with access to the hosting configuration, and a guess there reads as verified. A protocol permitting publication is a separate future protocol needing a consent mechanism that does not exist, not a version bump of this one.

Two schema changes were forced by writing it, both the same shape as permitted_uses. data.storage's three controls are now tri-state — a bare boolean gave an author who had not checked the hosting config only "claim true" or "claim false", and both assert something about an unexamined system. And participants.age_assurance.{method, jurisdiction} is now required: an inclusion criterion of "18+" does not say whether anyone checked, and jurisdiction rides with it because the operative consent age is set per member state, so a band without one decides nothing.

The pipeline reached the same rule independently, at row grain. experiments/learning-graph/report/gate.py in learning-engine-ai-frontend refuses publication from the compiled rows; this layer refuses a release from the protocol. Both are needed — a protocol can permit what the rows cannot support, and vice versa.

The two vocabulary mismatches are settled, and only one of them was a collision. pseudonymised was: one concept, two spellings, a join that breaks silently. This layer moved to the pipeline's spelling — ~21 occurrences in unmerged files here against 16 across six files there including a published JSON schema, eight test assertions and compiled Parquet, and GDPR's own term of art is "pseudonymisation" (Art 4(5)). Conceding on your own side is not the same act as changing the other repo by fiat. Do not rename it back.

The two source_type fields were never a collision, and this file said otherwise for an hour. They answer different questions: the pipeline's says what produced a row (human/synthetic/research/reconstructed), this layer's says what kind of record it is (research/runtime-learner-data/synthetic-simulation/llm-proposed/platform-experiment). Only research is shared, and human maps to either runtime-learner-data or platform-experiment depending on whether the release assigned conditions — which a row cannot know, because that is a property of the investigation. Renaming either side would destroy information. The identifiability scales only partly overlap for the same reason: identifiable (could be) is not identified (is), and anonymous is not deidentified.

So the crosswalk is code. python3 scripts/check_research.py --crosswalk prints it and exits 1 the moment the other side grows a value nothing maps — a crosswalk in prose rots. Where the scales differ the mapping is deliberately conservative, resolving toward the more identifiable reading, because it feeds a publication ceiling and the safe error is refusing a release that could have been allowed; None means the row does not determine this rather than a guess.

One synthetic fixture, marked at every level (source_type: synthetic-simulation, SYNTHETIC in every title and header comment): a protocol, two release versions, two evidence records, five reviews, one issue — and claims/spaced-practice-improves-long-term-retention.md is unchanged (it has since been merged into spaced-practice-improves-retention, which answers to the old slug as an alias). The evidence names it with a bearing and no anchor, so the edge exists and nothing fabricated appears in a real claim's ## Evidence section.

python3 scripts/check_research.py                     # validate; exit 1 on a problem
python3 scripts/check_research.py --summary           # what is in the layer
python3 scripts/check_research.py --issues            # issues, state and provenance
python3 scripts/check_research.py --why <claim-slug>  # the traversal
python3 scripts/lint.py --type research               # the same checks, in CI

The study plan is the fifth object, and it is what a reviewer reads¶

research/studies/<id>/<version>.yaml sits between the protocol and the release: Protocol → Study → Release → Evidence → Claim.

It is not a release, and that was forced rather than chosen. A release IS a publication — the layer refuses one whose protocol lacks permitted_uses['research-publication'] == permitted. A plan is not. Folding them together would either make it impossible to plan a study under a protocol that forbids publishing (exactly the internal pipeline-validation case lazuli-platform-telemetry describes) or weaken the publication gate. It is also the object an IRB actually reviews: they review a plan, not a paper.

That reasoning was tested, not assumed, and a parallel branch was rebuilt onto it. Another session had put pre-registration on the release as a stage: preregistration field. Planning an internal validation study under lazuli-platform-telemetry fails — "this release publishes findings, but its protocol's consent records permitted_uses['research-publication'] as 'prohibited'" — so the study is unplannable, and the message asserts it publishes findings about a file containing none. Do not reintroduce a stage: on the release. The two objects are not interchangeable and the publication gate is what the layer exists to enforce.

A primary endpoint requires prespecified_prediction: {prediction, disconfirming_result}. measure says what is measured and analysis_plan.tests says how it is analysed; neither says what result would count AGAINST the hypothesis, and without that any outcome can be narrated afterwards as a success. Optional on secondary and safety; refused on exploratory — an endpoint with a prediction is not exploratory, and carrying both is how an exploratory result is reported later as though it had been predicted. There is deliberately no mode: confirmatory | exploratory field: role already draws that line, and a second field saying the same thing is the drift shape this repo has lost weeks to on DOIs.

A release names the study it reports — study: {ref, version}, or no_study_reason: for a secondary analysis that genuinely had no plan of ours. The docstring claimed this edge from the start and nothing implemented it, so a plan and its report were two unconnected files. The plan's effective_from is checked against the release's released_at: a plan written after its own findings plans nothing, and only the dates reveal it.

analyses[].prespecified is required and never defaulted, and false is an ordinary honest value — a reviewer's question answered after the fact is good practice; what would not be is that analysis appearing with nothing to say it arrived late.

analyses[].endpoint_refs is declared, never inferred, and is a LIST. The first draft of --deviations matched endpoint names against analysis titles by word overlap and immediately reported an endpoint as unreported on a release that reports it — a fuzzy join that is wrong is worse than none, because the output looks like a finding. Where no analysis declares one, --deviations says endpoint coverage is not checkable rather than guessing. A list because one analysis routinely answers several endpoints.

python3 scripts/check_research.py --deviations <study-id>   # plan vs report
python3 scripts/check_research.py --stub-study <study-id>   # a plan skeleton

--deviations reports rather than judges. The spaced-review fixture's 1.1.0 adds time-on-task-model to answer a reviewer's issue, and that is what should happen.

A TODO anywhere in an observations/ or research/ file now fails validation, comments included. --stub promised this and delivered two-thirds: a marker in an enum or a date failed on its own shape, but one in free text validated clean — verified by injecting one into outcome.source_language and watching the whole store report zero. On a plan, free text is where disconfirming_result lives; a skeleton committable with its prediction still unwritten looks registered and is not.

Dates are validated as dates, after a draft protocol with effective_from: "NOT ESTABLISHED" validated clean. And PyYAML's timestamp constructor raises a bare ValueError on an impossible date, which crashed both checkers with a traceback instead of reporting one file; both loaders now catch it.

The study's own enrolment.special_populations was never shape-checked — only the protocol's — so a scalar there crashed check_study_against_protocol with an AttributeError. Found by typing special_populations: none, which reads as a perfectly reasonable thing to write. It is a mapping per class, because the protocol ceiling is checked per class and "which ones" is what a scalar cannot answer.

What lives on the study is what varies per study — the governing rule observations/ established. A protocol has no single question, no enrolment target and no endpoint, because it governs a family. So question, background, design, setting, resources, enrolment, timelines, procedures, endpoints, analysis_plan, benefits, participant_burden, results_sharing and community_involvement are the study's. Counts are {value, unit}: in a cluster design the number randomised and the number analysed are different numbers of different things.

A declared endpoint is not a measured one. endpoints[] says what the study set out to test; observations/ records what was measured. The difference between the two lists is what preregistration exists to police, and it is only visible because they are separate records. At least one endpoint must be role: primary.

conformance is machine-authored; determination is not, and they are separate blocks. A machine may compute that a plan satisfies a ruleset — naming the ruleset and version, or the result cannot be re-checked or superseded. A machine may never record that an IRB determined anything: determined_by must be human:<id> or org:<id>, the same discipline that makes append_authority() refuse a non-human verifier. And conformance.result must agree with its own checks — a conforming stamped over a failing check fails validation, because that summary is exactly what a reviewer would rely on.

status changes are version bumps, because the file is immutable. That is the feature: a material change writes a new version, which invalidates the conformance result computed against the old one. That is the escalation mechanism, not a workaround.

Three enforcement checks, protocol as ceiling and study as actual, each verified by mutating the fixture until it fired: a study may not enrol a special population its protocol does not permit (not-addressed is not permission); consent required by the protocol cannot be documented as none without a recorded waiver; and a study enrolling minors requires the protocol's consent.minors block.

Everything added to protocol is OPTIONAL, and that is forced by immutability — a newly required field would retroactively invalidate every frozen protocol. New: participants.special_populations, consent.process, consent.waiver (the five regulatory criteria as separate checkable fields), consent.minors, consent.withdrawal.investigator_initiated, data.subject_privacy, per-risk probability/magnitude/duration/reversibility, and sites.

What optionality buys is expressibility, not enforcement. Before these fields, "nobody established this" could not be said — the question had no home. The three pre-existing protocols went from 13–14 unanswerable sections to 0–1 without a single frozen file being touched; their gaps became unspecified rather than no-field.

data.subject_privacy is the one most easily got wrong. A reviewer's template is explicit that it is not storage: it is intrusiveness — how subjects are approached, how they are put at ease, and how the team is entitled to reach any source of information about them. Answering it from the storage and access fields answers a different question convincingly, which is worse than not answering it.

The synthetic fixture pair exercises the minors cluster on purpose, because minors are the population an automated review pathway is least likely to be able to cover: protocols/example-classroom-consented-study/1.0.0.yaml and studies/example-retrieval-practice-classroom/1.0.0.yaml, marked synthetic-simulation and SYNTHETIC throughout. Together they render every section of a reviewer's template: 26 filled, 4 not-applicable, 0 gaps.

Two YAML traps bit while writing them, both the same shape. A list item reading - Primary: arm coefficient… is a mapping, not a string, because of the :. The validator caught both (must be a non-empty string) — but a schema that accepted free -form values would have stored a dict where a sentence belonged and nothing would have said so.

Rendering a protocol as somebody else's form¶

scripts/render_brany_protocol.py renders a protocol (optionally with a release) as BRANY's SBER protocol template — the 26-section document an independent IRB would expect. It exists to answer one question checkably: which of an IRB's questions can this schema already answer, and which can it not?

python3 scripts/render_brany_protocol.py <protocol-id> [--release <id>] # the document
python3 scripts/render_brany_protocol.py <protocol-id> --coverage       # status per section
python3 scripts/render_brany_protocol.py <protocol-id> --gaps           # just what it cannot answer

A section the schema cannot answer renders as a loud marker, never as prose, and that is the whole discipline: a governance document filled in by inference reads as verified, which is the failure this file catalogues at length on DOIs. Five statuses, and the distinctions carry the meaning — unspecified (the schema has the field; this protocol has not established it) is not no-field (the schema has nowhere to put it), exactly as crossref_reachable: false is not flagged.

Some "not applicable" determinations are made by the record rather than by the author. BRANY requires a safety-monitoring plan and injury-compensation terms only above minimal risk, so risks.classification decides sections 16 and 23 — and not-determined correctly yields neither answer, because an undetermined classification decides nothing. That is a small worked example of the larger argument: a machine-readable protocol lets a reviewer's own rule run against the configuration.

Those gaps are closed, and the measurement is the record of it. When this was first run, 13 of 30 sections could not be answered at all (14 where risk classification was undetermined). The study object and the optional protocol fields above close every one: a protocol and a study plan together render 26 filled, 4 not-applicable, 0 gaps. The three frozen protocols, untouched, went from 13–14 unanswerable to 0–1 — their gaps are now unspecified, which is a statement, where before there was no field to leave empty.

Re-measure rather than trusting that paragraph:

python3 scripts/render_brany_protocol.py example-classroom-consented-study \
    --study example-retrieval-practice-classroom --coverage

The one section that still reported no-field did so correctly: section 23 on lazuli-platform-telemetry (now in the private repo), whose risks.classification is not-determined, so the rule that would make it inapplicable cannot fire. That is the record refusing to decide, not a missing field.

--why is the acceptance test run as a command: why does the wiki believe claim X, answered by walking claim ← evidence ← analysis ← dataset ← release ← protocol from the records alone, printing what is contested at every level, and naming any record whose source_type is not research loudly — so a simulation, runtime telemetry or a model's proposal is never mistaken for a published finding.

These are not wiki pages, deliberately: no type: frontmatter, no banner, no entry in wiki-index.json, the mkdocs nav or okf_lib.CONTENT_FOLDERS. Unattended batch tools iterate the content folders and rewrite what they find, and an immutable versioned object must not live where a batch job rewrites things. How a release is rendered is a separate problem and out of scope.

research/SCHEMA.md ends with the questions left deliberately unresolved — whether Analysis and Dataset become first-class objects, where deviations: belongs, whether ModerationDecision earns its own file, and the review-type vocabulary. Do not resolve them by guessing; resolve them when there are real releases.

Page identity — `id:` and `aliases:`¶

The learning-design-spec pipeline resolves design documents against this wiki: a pattern plan names element: <slug>, a principle names Realizes: <principle-slug>, a design section cites research:<claim-slug>, and spec/learners.md requires every learner dimension to be a learner-variables/ slug. Its contract (that repo's findings/0008) opens with every page's id is stable and unique within its kind — because a rename breaks a course that has already shipped, silently, outside this repo.

So the six kinds a design document can point at carry id: equal to their filename: elements, principles, patterns, claims, learner-variables, strategies. Theories stay out — nothing in the spec addresses one. A page type gains an id when something outside the wiki depends on pointing at it.

Strategies were nearly left out, and that would have been wrong. findings/0008's contract names only claim, pattern, element and principle slugs, and the reverse index reaches strategies without anyone naming them — so the first cut excluded all 2,557. spec/patterns.md then makes a phase name one outright:

phases:
  - phase: Read the case
    element: case-based-learning
    strategy: chunked-reading-with-embedded-questions   # a learning-wiki strategy slug

whose ### Instructions become that phase's authored brief. So a strategy rename breaks a shipped course exactly the way an element rename does.

python3 scripts/page_identity.py --check — report; --apply — stamp missing ids
python3 scripts/lint.py --type identity — id present, equal to the slug, and unique across ids and aliases within a kind

The claims id: was repurposed, deliberately. CLAUDE.md used to document it as a short programmatic code (we-4, fi-2). Measured before the change: present on all 422 pages and useless as identity — 56 blank, only 183 of the 366 non-empty equal to the slug, and six values shared by two pages each. enrich.py stamped an empty id: into every new claim, which is where the blanks came from. The short code was referenced nowhere in the wiki's 12,893 citations while the design spec addresses claims by slug throughout, so it was vestigial.

aliases: is what makes a rename non-breaking, and it is never stamped empty. It appears when a page is actually renamed: update_links_for_renames.py sets id: to the new slug and records the old one in aliases:, so a course document written against the old name still resolves. Empty fields on a thousand pages train a reader to skip the block.

Ids and aliases share one namespace per kind. A document saying element: foo cannot tell whether foo is a current slug or a retired one, so two pages answering to it is ambiguous however that came about — which is the case lint catches that filename-uniqueness never could, filenames being unique by construction.

A page moved to another kind keeps its old id through moved_from: (2026-10-20). Aliases live inside one kind, so a page folded or re-filed into another kind (a programme filed as a strategy, now a product) left strategy: good-behavior-game resolving to nothing. The page it went to lists the old kind-qualified ids, moved_from: [strategy/good-behavior-game]; build_wiki_index.py publishes them as a separate moved map in wiki-index.json ({old kind: {old slug: "new kind/new slug"}}, additive, so resolve keeps its shape), and the MCP resolve tool follows it. merge_pages.py writes it on every cross-kind fold. 1,570 retired ids were backfilled from git history and the merged-from comments; a page deliberately removed is never given a redirect.

Both YAML spellings are read (aliases: [a, b] and the block form), because both are written here — page_identity.py edits raw frontmatter text inline, while ingest_extractions.py builds pages through okf_lib.dump_frontmatter, which emits a block. A reader understanding only one would see no aliases on half the pages that have them: the rename would look recorded and still not resolve.

The spec reads exactly two things from wiki frontmatter: id and type:. Everything else it resolves is a body section — ### Instructions for a phase's brief, ### Target Learners and ### Target Learning Goals for matching personas and goals, #### Requirements/#### Constraints as the applicability filter, ## Claims with their inline markers, ## Examples on a learner-variable. Those are at 100% and ~99.8% already, and findings/0008 is explicit that they must stay prose: structuring them into YAML would lose the qualifications that make them useful. So there is no further frontmatter work for the pattern-building process — a pattern doc's pedagogy: block and a pack's elements: live in those documents, not here.

Note: verify_citation_edits.py will trip on an id/alias pass — those are frontmatter edits, not citation edits, and it is a guard for citation tool runs only.

The reverse index¶

reverse-index.json answers "which pages point at this one". It is generated by scripts/build_reverse_index.py and regenerated by build_indexes.py, so anything that adds or relinks a page already refreshes it; --check fails when the committed copy is stale.

It exists because the forward links from a principle are thin and the reverse links to it are not:

edge	forward	reverse
principle ↔ strategy	43 / 198 (22%)	1752 / 2557 (69%)
element ↔ strategy	—	2390 / 2557 (93%)
claim ↔ strategy	—	2324 / 2557 (91%)

Three times the coverage, and the more trustworthy direction: the author of a strategy knew which principle they were applying, while the author of a principle page is guessing at which strategies might exemplify it. So do not hand-populate ## Examples on 198 principle pages — one pass over the corpus gets more, and gets it right.

Claim edges carry the citation's polarity and strength from the inline marker, which is what lets the design side weight evidence per citation rather than per claim.

It is committed on purpose. CI on the design-spec side has no wiki checkout, so an index that only exists after someone runs a script is an index that side cannot use.

The wiki index¶

wiki-index.json is the table the learning-design-spec side resolves against: one record per content page — id, type, title, description, path, status, aliases, and grain_size where a page carries one — plus a precomputed resolve map per kind. scripts/build_wiki_index.py writes it and build_indexes.py regenerates it alongside the reverse index; --check fails when the committed copy is stale, and CI runs that check on every PR.

It exists because that repo shape-checks realizes: and cannot resolve it. A profile carries realizes: <pattern-slug> and a method realizes: <strategy-slug>, with the kind fixed by what carries the field; the same repo also resolves element:, research:<claim-slug> and spec/learners.md's learner-variable slugs. Its CI has no wiki checkout — that is a deliberate constraint on that side, not an oversight — so a typo'd slug passes CI, ships, and fails in front of a course author. A committed table is what closes that, the same arrangement and the same reason as reverse-index.json.

Aliases are the half a naive dump would omit and the half that matters. A rename is non-breaking only if the retired slug still resolves, and this file is where the spec side learns that 4cid means 4cid-four-component-instructional-design. Ship an index of current slugs only and every rename becomes a silent break one repo over — the exact failure aliases: exists to prevent, reintroduced at the boundary.

The resolve map is derivable and is emitted anyway. Ids and aliases share one namespace per kind, which is a rule a second implementation can get wrong; emitting the resolution once means every consumer resolves identically, and a collision is caught at build time by the side that can fix it. build_wiki_index.py refuses to write a table that resolves one slug two ways rather than letting a consumer pick whichever it saw last.

Both halves come out of one pass, so they cannot drift from each other. Theories are included even though nothing in the spec addresses one: a consumer filters on type for free, and a kind left out is a kind nobody can point at until somebody notices.

The MCP server — the same data, as tools¶

scripts/mcp_server.py serves the wiki over the Model Context Protocol, so an agent can query it without knowing how to read the index files. It has six read-only tools. search and fetch are in the shape deep-research clients expect, and ids are <kind>/<slug>. resolve handles current and retired slugs, and returns every kind a slug lives in. backlinks comes from the reverse index and carries each claim citation's [±~][SMW] marker. why is check_research.why() with its output captured, so the tool and the command cannot disagree. situations answers a learner and context analysis: given tags from evidence-dimensions.json (novice, online-self-paced, procedure, ...) or words, it returns the rows of every page's ## Fitting the design to a situation table that speak to them, each with its basis, from situation-index.json (scripts/build_situation_index.py, regenerated by build_indexes.py, --check in CI).

python3 scripts/mcp_server.py                         # stdio
python3 scripts/mcp_server.py --http 8765             # stateless HTTP, 127.0.0.1:8765/mcp
python3 scripts/mcp_server.py --call search '{"query": "retrieval practice"}'
claude mcp add learning-wiki -- python3 /path/to/learning-wiki/scripts/mcp_server.py

It depends on the standard library and PyYAML only. Both transports were tested against the official MCP Python SDK's client (2.2.0). Every page result carries status and trust_tier, and the server's instructions say the wiki is model-written. No page has a verified: entry, so an outside agent would otherwise take a fluent page for a checked one. The shape is borrowed from the Renaissance AI and Education Resource Hub's worker; no code was copied, because that repo has no licence yet. It is not hosted anywhere. It runs from a checkout, and nothing in it is newer than that checkout. Hosting it is a separate decision: GitHub Pages cannot run it.

Two kinds for the designer's side: `processes/` and `methods/`¶

Every other kind in this wiki describes instruction a learner meets. A pattern is a reusable design at lesson or unit level; a strategy is a concrete activity recipe. Design work is a different object entirely — what somebody does to produce the instruction — and it had been filed among them: ADDIE and design thinking sat in patterns/ beside cooperative learning, and task analysis sat in strategies/ beside think-pair-share, which advertises a designer's elicitation method as a teaching activity.

processes/ — whole design processes a designer works through: ADDIE and the Dick & Carey systems approach, SAM, design thinking, LXD, Understanding by Design, continuous improvement, faculty course design, Data Wise, humanizing co-design.
methods/ — the practices a process is made of: needs analysis, learner and context analysis, task analysis, learning-hierarchy task analysis, cognitive task analysis, learner personas, activity-system personas, backward design, standards crosswalk, family-interview assessment task design, making power dynamics visible.

The two levels are not cosmetic — they mirror the spec's own split. In learning-design-spec a profile realizes a whole process and a method realizes a practice, and the kind is fixed by what carries the field. One combined folder would have resolved both against one namespace, and nothing would catch a profile that named a crosswalk. Two kinds keep that error catchable.

realizes: therefore resolves against process and method now, not pattern and strategy. That is a change to the spec repo's decisions/0012, and it is not alias-able: aliases live inside one kind, so a retired kind cannot be aliased. Both sides have to move together, and wiki-index.json carries the resolve map for both new kinds so the spec side can do it in one edit.

The same slug in two kinds is fine and now happens on purpose. design-thinking is a process (the designer's cycle) and a strategy (students running the cycle on an authentic problem) — two genuinely different objects that share a name. Ids are unique per kind, so each resolves in its own namespace. The strategy page was briefly deprecated when the process needed the pattern namespace; it is restored.

The criterion is function inside learning design, not the discipline a method came from. Settled when stakeholder-mapping, logic-model and theory-of-change landed: all three were developed outside instructional design — in strategic management and in programme evaluation — and all three belong here, because a method is defined by the job it does in a design process, and each of them produces a named design artifact that a process profile requires. "Was it invented for instructional design" is the wrong question and would have excluded backward design's own ancestry. Note the asymmetry that keeps this from swallowing everything: the test is what the method produces in a design, so a general technique that produces nothing a design carries is still out.

What stayed put, and why, because these are the judgement calls:

strategies/task_analysis.md (underscore) is learners breaking an assignment into sub-tasks — a different practice from methods/task-analysis.md, not a duplicate of it, despite near-identical titles.
patterns/online-course-design.md, storyline-science-curriculum-design and interdisciplinary-course-based-research-experience describe the shape of the resulting course, not the steps to produce one. Patterns.
patterns/professional-development.md is instruction whose learners are teachers.
strategies/exemplar-analysis.md is learners studying worked examples.

Three are genuinely unresolved — flagged rather than moved:

strategies/formative-evaluation.md is classroom formative assessment, but instructional design uses the same words for testing draft materials with learners (one-to-one, small group, field trial). The process pages needed the second sense and four links were quietly asserting it; they were removed rather than left pointing at a page that says something else. methods/formative-evaluation-of-materials is a real gap.
The student-shadowing family (~10 near-identical pages) is educator-as-investigator work and belongs in methods/, but moving one of ten fragments the set. It is part of the near-duplicate backlog and should move as a group once deduplicated.
cede-power-student-voice and student_feedback_on_the_class sit on the line between a design stance and a teaching practice. Left as strategies.

Adding a kind touched fourteen lists, which is thirteen too many. Not a figure of speech: the folder list was hard-coded in fourteen places, and lint.py's own copy had never contained learner-variables — so for as long as that folder has existed no page in it was checked for a broken link, and the "N pages indexed" line was quietly twelve short. It reported OK because it never looked. The copies that mean "every content folder" now derive from okf_lib.CONTENT_FOLDERS; the ones carrying per-kind data (labels, singulars, index config) must stay hand-written, so check_nav_coverage fails when a content folder is missing from any of them, naming the specific consequence.

Page templates: a process uses the Pattern template (it has a Sequence, Requirements and Constraints, a grain size) and a method uses the Strategy template (it has Instructions). Those section names are deliberately unchanged — ### Instructions, ### Target Learners, ### Target Learning Goals, #### Requirements/#### Constraints are what the spec reads out of the body, and renaming them to suit the new folder would break the thing the new folder exists to serve. Only ## Related Patterns / ## Related Strategies become ## Related Processes / ## Related Methods, which nothing reads.

Evidence markers are the evidence layer¶

The [±~][SMW] marker beside a claim link is the one field the design-spec pipeline reads for evidence quality. It beats the frontmatter evidence_strength: in a way worth naming: strength attaches to the citation, not the claim, so a claim cited [+S] on one page and [~W] on another records a disagreement one number per claim would flatten. Polarity carries as much as strength — a principle citing [-S] cites strong evidence against, which is a constraint on the principle, not support for it.

python3 scripts/check_evidence_markers.py reports citations without one: 244 across 196 pages, out of 11,697 outside claims/ — 97.9% already carry one. Links from claims/ are excluded, because a claim linking a claim is a ## Related Claims relation rather than a citation (247 of the 491 raw unmarked links are that).

It is a script, not a lint check, and deliberately so. Adding it to lint.py would take it from 0 to permanently red, destroying the property this file leans on — that a count of zero means the work is done rather than the check being broken. find_title_duplicates.py sets the precedent. And the backlog cannot be cleared mechanically: a marker asserts a polarity and a strength about how this page uses this claim, so deriving it from the claim's frontmatter, or copying the majority marker other pages used, would manufacture exactly the kind of plausible unverified detail this repo already lost weeks to on DOIs.

Do not clean up evidence_strength:. 19 distinct values across 422 claims, 117 blank, nine of them spellings of "not yet". The inline notation supersedes it and the design side does not read it.

Three core operations¶

1. Ingest¶

Process a new source (paper, book chapter, CSV batch, worked example) into wiki pages.

Steps: 1. Identify the page type(s) the source contributes to (principle, element, pattern, strategy, theory, learner-variable, claim). learner-variable is schema-ready but not yet part of the automated single-pass extraction prompt (deliberately deferred to a dedicated future sweep, so the extraction agent isn't juggling a fourth job on top of claims/omission/fabrication) — for now, factor a learner-variable page out by hand when a claim reports a finding about a learner characteristic (e.g. "X predicts/moderates Y outcome"), rather than leaving it as a bare, unlinked claim. 2. Check if a page already exists (index.md or grep by name) 3. If new: create a page in the correct folder using the template below 4. If existing: merge new content into the right sections; append to ## Key Sources (or ## Evidence for claims); log the change 5. Cross-link: add markdown links like [Example Page](../principles/example-page.md) to related pages already in the wiki 6. Update index.md — run python3 scripts/build_indexes.py to regenerate it and every per-folder index from disk state 7. Append an entry to log.md under today's ## YYYY-MM-DD heading: * **Ingest**: [page](folder/page.md) — [source] (or run python3 scripts/log_revision.py <page> --by <actor> --type ingest --desc "...", which updates the page's generated field, its revision card, and log.md in one step)

2. Query¶

Answer a question by reading the wiki.

Steps: 1. Search index.md for relevant pages 2. Read those pages; follow the markdown links (slug.md / ../folder/slug.md) as needed 3. Synthesize across pages; cite page names and claim IDs 4. Flag gaps: if the answer requires a page that doesn't exist, note it

3. Lint¶

Health-check the wiki: python3 scripts/lint.py [--fix].

Checks: - Broken cross-links (slug.md / ../folder/slug.md link target not found) - Pages with status: draft and no description - Claim pages missing an evidence strength rating - Principles missing at least one claim link - Claim evidence entries missing a DOI or URL

Evidence tags¶

Used inline in principle, pattern, element, and strategy pages when citing claims. The tag describes the direction of the claim's effect on the page's topic, not just evidence strength.

Tag	Meaning
[+S]	Supports — strong (consistent experimental/meta-analytic)
[+M]	Supports — moderate
[+W]	Supports — weak / emerging
[~S], [~M], [~W]	Contextual / mixed — effect depends on conditions (e.g. expertise reversal: works for novices, not experts)
[-S], [-M], [-W]	Contradicts or reduces effectiveness — strength varies
[X]	Contradicted / discredited

Rule: Claims cited in a Constraints section should use [-] (negative effect) or [~] (contextual/mixed), never [+]. A constraint describes a condition where the approach fails or causes harm — the tag should reflect that direction, even if the underlying claim is phrased positively (e.g., "practice improves transfer" cited as evidence that lack of practice hurts outcomes → [-S]).

Always link the tag to a claim page: [Claim statement](../claims/example-claim.md) [+M]

Relation symbols (cross-link annotations)¶

+ supports / reinforces
~ contextual / mixed / depends on conditions
– contradicts / undermines

Status values¶

Status	Meaning
`draft`	Skeleton or stub; content not reviewed
`review`	Content present; needs expert review
`stable`	Reviewed and considered reliable
`deprecated`	Superseded or discredited; kept for history

Frontmatter fields¶

Every content page (principle, element, pattern, design, product, strategy, process, method, research method, theory, learner-variable, claim) carries this OKF-conformant frontmatter:

Field	Required	Meaning
`type`	Yes	`principle` \| `element` \| `pattern` \| `design` \| `product` \| `strategy` \| `process` \| `method` \| `research-method` \| `theory` \| `learner-variable` \| `claim`
`title`	Recommended	Display name — normally matches the page's `# H1`
`description`	Recommended	One-sentence summary, used in index listings
`status`	Recommended	See Status values above
`generated`	Recommended	`{ by: <actor>, at: <date> }` — who/what last wrote the page and when, replacing the old `last_edited`/`edited_by` pair
`sources`	When applicable	List of `{ id, resource, title, author, q, i, n }` entries parsed from `## Key Sources` (or `## Evidence` for claims) — a structured mirror of the citations already in the body, not a replacement for them
`verified`	Optional	List of `{ by: <actor>, at: <date> }` confirmation events — see Trust tiers below. Absent on every page until someone explicitly reviews it; never set this yourself just because a page looks complete
`id`	On identified kinds	Equal to the filename slug, on elements/principles/patterns/claims/learner-variables — the stable name design documents resolve by (see Page identity below)
`aliases`	After a rename	Every former slug of this page, retained forever, so a document naming the old one still resolves
`evidence_strength`, `author`, `grain_size`	Type-specific	Extra scalar fields kept as-is per page type (see templates below); OKF tolerates extra frontmatter keys

Actor convention for generated.by (and any other identity field): <tool>/unspecified for an agent/tool (e.g. claude/unspecified, codex/unspecified), human:<id> for a person, process:<id> for an unattended batch job (e.g. process:wiki-ingest). Never invent a specific model version you're not certain of — unspecified is fine.

Trust tiers (`verified`)¶

verified is a different axis from evidence_strength, and the two should never be conflated:

evidence_strength (and the per-study q/i codes in a claim's ## Evidence section) describe how strong the underlying research is — is this a meta-analysis or a single case study.
verified describes whether a human has actually checked that this wiki page faithfully represents that research — a claim can have evidence_strength: strong and still be completely unverified, because the LLM that wrote the page could have paraphrased a finding wrong, mistagged an effect's direction, or introduced a citation error that nothing has caught yet.

OKF derives three trust tiers from the verified field:

Tier	Condition
unverified	No `verified` key present (the default for every freshly ingested or LLM-edited page)
machine-confirmed	`verified` present, but only by non-`human:` actors (e.g. a lint pass)
human-reviewed	`verified` present with at least one `human:<id>` actor

Format (a list, so repeated reviews over time each add an entry — OKF also tolerates a bare single mapping):

verified:
  - by: human:david
    at: 2026-08-22

Add a verified entry when a human substantively reviews a page's content for accuracy — not on every procedural PR approval. The easiest way is:

python3 scripts/log_revision.py <page> --by human:<id> --type status --desc "Reviewed for accuracy" --verify

which appends the verified entry alongside its normal generated/log-update work. Never add a verified entry yourself (as an agent) just because a page looks complete or well-sourced — that defeats the point of the tier. python3 scripts/lint.py flags any status: stable page that has no verified entry, since "stable" should mean someone actually checked it, not just that it looks finished.

Every content page carries a one-line banner directly under its # H1, naming the page's type and linking back to its section index:

# Cooperative Learning

> **Principle** · [All principles](index.md)

This exists because 73 slugs live in more than one type folder — cooperative-learning and direct-instruction each exist in all four of principles/elements/patterns/strategies, with near-identical titles. Frontmatter carries type, but mkdocs strips frontmatter out of the rendered page entirely, so on the docs site, in GitHub's file view, in the dashboard's edit box, and in whatever an agent reads during an ingest, the folder in the URL was the only thing distinguishing them. A blockquote renders as a visible callout in both GitHub and mkdocs-material.

The banner's label follows the folder the page is in, which is what actually determines its section — so where frontmatter type and the folder disagree, that's a real data bug (the page is either misfiled or mislabelled) and a human decides which. lint.py's check_type_banner verifies all three agree — banner present, label matches folder, and frontmatter type matches folder — on every health run.

Run python3 scripts/add_type_banner.py --apply after any batch that creates pages; it's idempotent (updates an existing banner in place rather than stacking a second one), so it's safe to re-run at any time. --check reports without writing.

Yes, this duplicates type: from frontmatter into the body — the same tradeoff this schema already accepts for sources: mirroring the citations in ## Key Sources: a structured field and a human-readable rendering of the same fact, kept in sync by a lint check rather than by dropping one.

Seeing the rest of the frontmatter¶

The banner covers one field. docs_hooks/page_metadata.py — a native mkdocs hook, so no new dependency — appends a collapsed Page metadata panel to every content page with the whole block: type, id, aliases, status, who generated it and when, the sources count and ids, and anything else the page carries, including fields added to the schema later.

It also renders the trust tier that CLAUDE.md derives from verified but nothing has ever displayed — unverified / machine-confirmed / human-reviewed. That turned out to matter immediately: no page in the wiki carries a verified: entry. The only match in the repo is the example in this file. Every one of the 3,721 content pages reads "unverified — no one has confirmed this page's content", which is accurate and was previously invisible.

Index pages, log.md and this file are skipped, and so is any page without a type: — those are OKF-reserved or not content.

Renaming a page¶

Renaming a page is two jobs, and git mv only does the first. Nothing else in the wiki updates the pages that link to the one you moved, so a rename lands as a set of silently broken cross-links.

The dangerous version is a rename that is correct in its own tree and breaks links only when merged forward, because the branch was cut before the linking pages existed. That has already happened once here: merging the scraper branch orphaned 17 links to a_finder's_guide_to_facts.md, renamed there to a_finders_guide_to_facts.md.

After renaming, always run:

python3 scripts/update_links_for_renames.py --from-git --dry-run
python3 scripts/update_links_for_renames.py --from-git --apply

It reads the renames staged in git (so git mv first, or git add the rename), and re-points every inbound link — same-folder, ../folder/, bundle-absolute, and the percent-encoded spellings models emit for slugs containing ', ", ?, ,, (, ), &, +. Every rewrite is checked to resolve on disk before it is kept, so the pass cannot turn a working link into a broken one. Pass --map <file> instead of --from-git to drive it from an explicit old<TAB>new list.

Then confirm with python3 scripts/lint.py --type broken_links and regenerate indexes with python3 scripts/build_indexes.py.

Cross-link conventions¶

Cross-links are standard markdown links, relative to the linking page: slug.md for another page in the same folder, ../folder/slug.md for a page in a different folder (every content folder sits exactly one level under the wiki root, so ../folder/ always resolves correctly regardless of which folder you're linking from)
OKF also permits absolute bundle-relative paths (/folder/slug.md) — this wiki uses the relative form instead because it works with plain mkdocs (the docs site's builder) with no extra plugin, whereas an absolute path renders as a literal domain-root URL once the site is hosted under a subpath
Slugs are lowercase, hyphen-separated: worked-examples, cognitive-load-theory
Always include the folder in a cross-folder link so the target is unambiguous: [Worked examples reduce novice search](../claims/worked-examples-reduce-novice-search.md)
Claims use semantic slugs: ../claims/worked-examples-reduce-novice-load.md, and id: in frontmatter equals that slug (see Page identity)
A link to a page that doesn't exist yet is tolerated (OKF requires consumers to tolerate broken links) — write the link anyway rather than leaving a bare TODO if you know the target slug, but don't invent slugs you haven't verified exist or are about to create

Folder map¶

ld-wiki/
  CLAUDE.md          ← this file (schema + operating guide)
  index.md           ← OKF bundle-root index; carries okf_version in frontmatter
  reverse-index.json ← generated: which pages point AT each page (see The reverse index below)
  wiki-index.json    ← generated: id + kind + title per page, for the design-spec side (see The wiki index below)
  evidence-scales.json ← what q3 / i2 / n= mean, as data (read by agents and by the docs hook)
  evidence-dimensions.json ← the shared dimensions of evidence and design (learner state, goal, outcome, ...)
  evidence.md        ← generated: the state of the wiki's evidence (build_evidence_report.py)
  log.md             ← reserved OKF filename: append-only, date-grouped change log (current month)
  log/               ← earlier months of log.md, one file each (okf_lib.rotate_log)
  history/agent-log.md ← dated entries: what each wiki-wide change did and why (newest first)
  principles/        ← design principles (what to do and why)
  elements/          ← instructional components (building blocks)
  patterns/          ← instructional patterns (reusable designs at lesson/unit level)
  designs/           ← designs for one setting, course or population, using patterns
  products/          ← named products and programmes others adopt, with their own frameworks and evidence
  strategies/        ← teaching strategies (concrete activity recipes)
  processes/         ← design processes: how a course gets made (ADDIE, SAM, design thinking, UbD)
  methods/           ← design methods: the practices a process is made of (task analysis, personas, crosswalks)
  research-methods/  ← methods for studying learning and evaluating education (education-specific only)
  theories/          ← learning theories (explanatory frameworks)
  learner-variables/ ← canonical learner characteristics (prior knowledge, self-efficacy, ...) claims link into
  claims/            ← empirical claims with evidence
  observations/      ← structured record of what each study actually did (see above)
  research/          ← the research layer: where knowledge came from (see above)
    protocols/<id>/<version>.yaml   ← versioned, immutable governance configuration
    studies/<id>/<version>.yaml     ← versioned, immutable STUDY PLANS (see below)
    releases/<id>/<version>.yaml    ← versioned, immutable investigations
    reviews/<review-id>.yaml        ← structured contributions interrogating any of it
    issues/<issue-id>.yaml          ← one problem, however many people found it
  sources/           ← bibliographic source pages (optional; most citations live inline in Key Sources / Evidence)
    manifest.ndjson    ← append-only log of every source reviewed, ingested or rejected (see Source Manifest below)
    authorities.ndjson ← append-only log of citations a HUMAN verified against the real source (see Citation Authorities below)
  scripts/
    okf_lib.py         ← shared OKF helpers (frontmatter parse/dump, link conversion, actor formatting)
    ingest.py          ← CSV → wiki pages batch ingest
    enrich.py          ← LLM-based stub enrichment (Claude/Gemini)
    build_indexes.py   ← regenerates index.md and every per-folder index from disk state
    log_revision.py    ← records a revision card + updates a page's `generated` field + appends to log.md
    log_source_review.py ← appends one entry to sources/manifest.ndjson (see Source Manifest below)
    add_type_banner.py ← inserts/refreshes the page-type banner under each page's H1 (see below)
    add_evidence_summary.py ← the `> **Evidence** ·` header line on claim pages (see below)
    update_links_for_renames.py ← after pages are renamed, re-points every inbound cross-link (see below)
    page_identity.py   ← stable `id:`/`aliases:` for the kinds design docs point at (see below)
    build_reverse_index.py ← who points at this page, as shippable data (see below)
    build_wiki_index.py ← the resolution table learning-design-spec reads (see below)
    check_evidence_markers.py ← claim citations carrying no [±~][SMW] marker (see below)
    evidence_rollup.py ← distinct-study roll-ups behind the evidence profiles and evidence.md
    add_evidence_profile.py ← the `> **Evidence** ·` line on pages that cite claims
    build_evidence_report.py ← evidence.md, the state of the wiki's evidence (generated)
    search_index.py    ← SQLite FTS5 index of the wiki in .cache/, used by search and linking
    link_claims.py     ← labels and writes Related Claims links between claims
    merge_claims.py    ← folds a duplicate claim into its canonical page, keeping an alias
    link_pages.py      ← links theories, principles, patterns, elements, strategies into the graph
    check_load_bearing.py ← judges the most-cited claims' evidence against their sources; writes nothing
    health_evidence.py ← the evidence and link section of the health report (load-bearing, single-study, i codes)
    recode_impact.py   ← recodes an i code to what the source prints, verified verbatim, else i?
    smd_worklist.py    ← WWC/ESSA review pages to extract next, for the SMD family (see above)
    priority_worklist.py ← what to extract next: cited-but-unrecorded, hub, and evidence-less claims (see above)
    mcp_server.py      ← the wiki as MCP tools: search, fetch, resolve, backlinks, why (see above)
    observation_lib.py ← the evidence layer's schema and validator (see above)
    research_lib.py    ← the research layer's schemas, validators and joins (see above)
    check_research.py  ← validate the research layer; --why traverses claim -> protocol
    lint.py            ← health-check (see Lint above)
    verify_citation_edits.py ← after a citation tool writes: did it edit only citations? (see above)
    check_embargo.py   ← refuses to push embargoed research into this PUBLIC repo (see settled items)
    mint_release_doi.py ← reserve, then publish, a Zenodo DOI for a research release (research/PUBLISHING.md)
    build_site.sh      ← the docs site: mkdocs (curated tier), build_longtail.py (the rest), Pagefind
    build_longtail.py  ← renders claims, strategies and non-canonical elements/theories as light pages
  docs_hooks/        ← mkdocs hooks: page_metadata.py (metadata panel), two_tier.py (which tier a page is in)

Each folder's index.md is itself a reserved OKF filename: no frontmatter (except the bundle-root's okf_version), and a plain * [Title](slug.md) - description bullet listing grouped by status. Regenerate these with python3 scripts/build_indexes.py rather than hand-editing them.

Source Manifest¶

sources/manifest.ndjson is an append-only record of every source article the ingest pipeline has reviewed — whether it contributed pages or was rejected as out of scope. It exists so anyone (including people outside this project) can check whether a specific article has already been covered, or audit the whole scan, at a scale (eventually tens of thousands of articles) where a rendered list or one page per source stops being practical. It is not meant to be human-browsed — it's a data file, not a wiki page.

Format: one JSON object per line (NDJSON), never rewritten or reordered — only appended to.

{"id": "eric-ed265520", "title": "The Effects of High and Low Relevant Text Underlining on Test Performance.", "doi": null, "reviewed_at": "2026-08-27", "status": "ingested", "pages": ["elements/text-underlining-and-annotating.md", "theories/von-restorff-effect-text-marking.md"]}
{"id": "eric-ed616622", "title": "A Bibliography of Cognitive Information Processing Theory, Research, and Practice", "doi": null, "reviewed_at": "2026-08-27", "status": "rejected", "reason": "matched the search topic on keyword overlap only, but the source is a career/vocational-counseling bibliography, not a learning-science theory; out of scope for this wiki"}

Fields: id (source identifier — the ERIC/PMC/arXiv id from the automated pipeline, or doi:<doi> / a URL for manually-ingested articles), title, doi (nullable), reviewed_at (ISO date), status ("ingested" or "rejected"), and either pages (bundle-relative paths the source contributed to, for "ingested") or reason (why it didn't contribute, for "rejected").

"rejected" entries also carry reason_code, one of a closed set in okf_lib.REJECTION_CODES: opinion-piece, out-of-scope, no-ingestable-content and already-covered are verdicts about the source, and map one-to-one onto the exclusions E1–E4 in INCLUSION.md (approved by the maintainer 2026-09-24). Every research paradigm is eligible there, and domain or setting is never a reason to exclude; parse-error, validation-error and no-contributions-extracted are failed runs that say nothing about it. The code is what makes rejections countable, and it is what discovery reads: discover_articles.load_excluded_ids() now skips every source the manifest has settled, which it did not before. It used to skip only the ten benchmark articles and the ids in eval/corpus/processed_articles.json, and that registry is not committed. So a discovery run on any other machine re-surfaced, and paid to re-extract, sources already ingested or rejected. A failed run stays eligible, so the registry's bounded retry still decides it. The 27 rejections written before 2026-09-24 have no code and are not being backfilled, because the file is append-only. They are treated as verdicts, except the three exact strings ingest_extractions.py wrote for its machine failures, which are recognised by that wording. log_source_review.py requires --reason-code with --reason, and prints any earlier entry for the same id or DOI before writing. A second review is still written, because a re-review is legitimate, but it has to be a choice. lint.py fails on a code outside the set.

"ingested" entries also carry citations: {checked, crossref_reachable, removed, flagged} — what the citation gate found on the pages that source wrote. removed lists DOIs stripped because they resolved to the wrong paper; flagged lists findings left for a human (a DOI on two papers, invented journal metadata, an invented title). crossref_reachable: false means the network check could not run, so that line is an unverified ingest rather than a clean one — never read a bare "ingested" as "citations were checked".

Always append via the helper, never hand-edit the file:

python3 scripts/log_source_review.py --id "doi:10.1234/example" --title "Article Title" \
  --status ingested --pages claims/foo.md elements/bar.md

python3 scripts/log_source_review.py --id "doi:10.1234/other" --title "Other Article" \
  --status rejected --reason-code out-of-scope --reason "not learning-science, out of scope"

(scripts/ingest_extractions.py, the automated eval-pipeline ingest path, calls okf_lib.append_manifest_entry() directly instead of shelling out to this script — same effect.)

Looking something up (no need for a script — it's just NDJSON):

grep '"id": "eric-ed265520"' sources/manifest.ndjson
grep -i '"title":.*underlining' sources/manifest.ndjson
python3 -c "import json,sys; [print(l) for l in map(json.loads, open('sources/manifest.ndjson')) if l['status']=='rejected']"

Known gap: the CSV batch-import path (scripts/ingest.py, reading external ~/research_briefs/*.csv files) doesn't write to the manifest — those rows have no natural per-article identity to key an entry on.

Citation Authorities — the human's side of the record¶

sources/authorities.ndjson is where a person records what they checked against the real source: the publisher's page, the book in hand, the article's own PDF. It is append-only NDJSON like manifest.ndjson, keyed by the same author-year key check_citations.py uses.

It exists because every automated check in this repo verifies against Crossref, and Crossref can only see what it indexes. 2,493 of the wiki's 12,893 citations — 1,232 distinct author-year keys — carry no DOI and no journal metadata at all. They are books, and all five checks are structurally blind to them: every one needs two variants of something to compare, and a book citation offers nothing to compare. Nothing has ever verified that Ambrose et al. (2010) is How Learning Works, Jossey-Bass, ISBN 978-0-470-61760-1. No lookup would say so. A person can.

python3 scripts/log_authority.py --key ambrose-2010 --by human:david \
  --title "How Learning Works: Seven Research-Based Principles for Smart Teaching" \
  --publisher "Jossey-Bass" --isbn 978-0-470-61760-1 --url https://www.wiley.com/... \
  --note "Verified against the Wiley product page."

python3 scripts/apply_authorities.py --check     # then --apply
python3 scripts/citation_worklist.py             # what to look at next, ranked

Four things about it that are deliberate:

An agent must never author one. append_authority() refuses any verified.by that is not human:<id>, for the same reason CLAUDE.md forbids an agent adding a page's verified: entry because the page looks complete. A machine-written "authority" is just another unverified assertion wearing a badge that says otherwise.
"doi": null is a verdict; an absent doi field is not. The first says a human established that no DOI is registered, which makes any DOI a later batch invents for that key provably wrong and strippable without a lookup. The second says only "not established", and nothing may act on it. Same discipline as crossref_reachable: false vs flagged, and as error vs wrong_paper.
It ratchets. lint.py's check_authority_conflicts fails on any citation that contradicts a recorded verdict, so a settled decision survives the next enrichment batch. Repairing the pages alone does not: the model that invented a DOI for a book once will invent one again, and nothing in the corpus remembers otherwise.
Only the no-DOI verdict is auto-repaired. apply_authorities.py reports a wrong ISBN, a differing DOI or a mismatched title rather than rewriting it — the page may be citing a different edition, a chapter rather than the book, or a reprint, and only the person who checked can say which. The tooling has been confidently wrong about exactly this kind of "obvious" correction before (the Cook-Sather title case above).

scripts/citation_worklist.py is the reading end. It collapses all five checks onto the author-year key — so several report lines about one source appear once — ranks by how many citations ride on it, ranks the book backlog separately (different work: a person and a publisher page, not a lookup), and drops keys already settled, so the list shrinks as you work.

Page templates¶

Principle¶

---
type: principle
id: [principle-slug]      # equal to the filename
title: [Principle Name]
description: [One-sentence summary of the recommendation]
status: draft
generated:
  by: <actor>
  at: YYYY-MM-DD
---

# [Principle Name]

> **Principle** · [All principles](index.md)

## Description
[What this principle is and what it recommends.]

## Implications

### Context
#### Requirements
- 
#### Constraints
- 

### Target Learners
- 

### Target Learning Objectives
- 

### Theory
#### Supporting
- 
#### Contradicting / Qualifying
- 

### Claims
<!-- Link claims with evidence tags: [Claim statement](../claims/claim-slug.md) [+M] -->
- 

## Related Principles
- 

## Examples
<!-- Links to elements or patterns that apply this principle -->
- 

## Key Sources
-

Element¶

---
type: element
id: [element-slug]      # equal to the filename
title: [Element Name]
description: [One-sentence summary of what this element is]
status: draft
generated:
  by: <actor>
  at: YYYY-MM-DD
---

# [Element Name]

> **Element** · [All elements](index.md)

## Description
[What this instructional element is; how it functions.]

## Design Implications

### Context
#### Requirements
- 
#### Constraints
- 

### Target Learners
<!-- Link to sub-claims: [Claim statement](../claims/claim-slug.md) -->
- 

### Target Learning Goals
<!-- Link to sub-claims: [Claim statement](../claims/claim-slug.md) -->
- 

### Affordances
<!-- Link to principles applied: [Principle Name](../principles/principle-slug.md) -->
- 

### Claims
<!-- Claims bearing on this element, each a link to a claim page followed by its evidence tag, e.g. [+M] -->
- 

## Related Elements
- 

## Examples
<!-- Links to strategies that use this element, with ratings -->
- 

## Key Sources
-

Pattern¶

---
type: pattern
id: [pattern-slug]      # equal to the filename
title: [Pattern Name]
description: [One-sentence summary of what this pattern is]
status: draft
generated:
  by: <actor>
  at: YYYY-MM-DD
author: 
grain_size: 
---

# [Pattern Name]

> **Pattern** · [All patterns](index.md)

## Description
[What this pattern is; how it works; what problem it solves.]

## Implications

### Context
#### Requirements
- 
#### Constraints
- 
#### Grain Size
[program / course / unit / lesson]

### Target Goals
<!-- Link to claims: [Claim statement](../claims/claim-slug.md) -->
- 

### Target Learners
<!-- Link to claims: [Claim statement](../claims/claim-slug.md) -->
- 

### Theory
#### Supporting
- 
#### Contradicting / Qualifying
- 

### Claims
<!-- Link claims with evidence tags -->
#### Supporting
- 
#### Contradicting
- 

## Design

### Sequence
<!-- Steps with links to elements: [Element Name](../elements/element-slug.md) -->
1. 

### Affordances
<!-- Links to principles applied: [Principle Name](../principles/principle-slug.md) -->
- 

### Personalization
<!-- How to adapt for non-target learners -->
- 

## Related Patterns
- 

## Examples
<!-- Links to products / lessons / courses with ratings -->
- 

## Key Sources
-

Design¶

A design for one setting, course, population or product: a self-paced mobile Italian A1 course, a clinical onboarding programme, one study's lesson sequence. It is never general (that is a pattern); it says what it uses the general patterns for, given its learners, goals and constraints.

---
type: design
id: [design-slug]      # equal to the filename
title: [Design Name]
description: [One sentence: for whom, in what setting, to what end]
status: draft
generated:
  by: <actor>
  at: YYYY-MM-DD
author:
grain_size:            # program / course / unit / lesson
---

# [Design Name]

> **Design** · [All designs](index.md)

## Description
[What the design is, who it is for and where it runs.]

## Context
### Learners
- 
### Learning Goals
- 
#### Requirements
- 
#### Constraints
- 

## Patterns Used
<!-- Each pattern the design applies, and what for: [Pattern Name](../patterns/pattern-slug.md): ... -->
- 

## Sequence
<!-- Steps with links to elements and strategies -->
1. 

### Claims
<!-- Claims bearing on this design's choices, with evidence tags -->
- 

## Related Designs
- 

## Key Sources
-

Product or Programme¶

A named product or programme others adopt or use. product_kind is one of software, curriculum, assessment, dataset, programme, framework. Its own frameworks, tools and indicators are components on this page, not pages of their own.

---
type: product
id: [product-slug]      # equal to the filename
title: [Name, as its maker names it]
description: [One sentence: what it is and who makes or runs it]
product_kind: programme
status: draft
generated:
  by: <actor>
  at: YYYY-MM-DD
---

# [Name]

> **Product or Programme** · [All products and programmes](index.md)

## Description
[What it is, who makes or runs it, who it is for, what it is meant to achieve.]

## Components
<!-- Its own frameworks, tools, indicators and instruments, as its sources describe them -->
- 

### Claims
<!-- Claims about it or tested with it, with evidence tags -->
- 

## Related Products and Programmes
- 

## Key Sources
-

Research Method¶

A method for studying learning or evaluating education that is specific to education or especially useful there. A general social-science method is not a page.

---
type: research-method
id: [method-slug]      # equal to the filename
title: [Method Name]
description: [One sentence: what question it answers and how]
status: draft
generated:
  by: <actor>
  at: YYYY-MM-DD
---

# [Method Name]

> **Research Method** · [All research methods](index.md)

## Description
[What the method is, what question it answers, what data it needs, and its known limits.]

## Accounts
<!-- How each source describes or uses the method -->
- 

### Claims
- 

## Related Research Methods
- 

## Key Sources
-

Strategy¶

---
type: strategy
id: [strategy-slug]      # equal to the filename
title: [Strategy Name]
description: [One-sentence summary of what this strategy is]
status: draft
generated:
  by: <actor>
  at: YYYY-MM-DD
---

# [Strategy Name]

> **Strategy** · [All strategies](index.md)

## Description
[What this strategy is and how it is carried out.]

## Design Implications

### Context
#### Requirements
- 
#### Constraints
- 
#### Implementation Variability
- 

### Target Learners
<!-- Link to sub-claims: [Claim statement](../claims/claim-slug.md) -->
- 

### Target Learning Goals
<!-- Link to sub-claims: [Claim statement](../claims/claim-slug.md) -->
- 

### Instructions
<!-- Steps with links to elements: [Element Name](../elements/element-slug.md) -->
1. 

## Related Strategies
- 

## Examples
<!-- Links to products with ratings -->
- 

## Key Sources
-

Theory¶

---
type: theory
title: [Theory Name]
description: [One-sentence summary of what this theory proposes]
status: draft
generated:
  by: <actor>
  at: YYYY-MM-DD
---

# [Theory Name]

> **Theory** · [All theories](index.md)

## Description
[What this theory proposes; its core mechanism or claim.]

## Implications

### Context
- 

### Target Learners
- 

### Target Learning Objectives
- 

## Claims
<!-- Claims that derive from or test this theory: [Claim statement](../claims/claim-slug.md) [+M] -->
- 

## Related Theories
- 

## Examples
<!-- Links to patterns and principles that apply this theory -->
- 

## Key Sources
-

Learner Variable¶

A canonical page per distinct learner characteristic/variable (prior knowledge, self-efficacy, working memory capacity, spatial ability, ...). Claims that report a finding about the variable link into it, the same way claims link into theories — this keeps "prior knowledge," "prior domain knowledge," and "background knowledge" from three different articles as one page instead of three fragmented, undiscoverable mentions. Schema-ready but not yet part of the automated single-pass extraction prompt — see the Ingest section above for why, and factor these out by hand for now when a claim clearly reports a learner-characteristic finding.

---
type: learner-variable
id: [variable-slug]   # equal to the filename
title: [Variable Name]
description: [One-sentence definition of this learner characteristic]
status: draft
generated:
  by: <actor>
  at: YYYY-MM-DD
---

# [Variable Name]

> **Learner Variable** · [All learner variables](index.md)

## Description
[What this learner variable is; how it's typically measured or operationalized.]

## Implications

### Context
- 

### Target Learners
- 

### Target Learning Objectives
<!-- Learning outcomes this variable has been shown to affect -->
- 

## Claims
<!-- Claims reporting findings about this variable, with evidence tags: [Claim statement](../claims/claim-slug.md) [+M] -->
- 

## Related Learner Variables
- 

## Examples
<!-- Links to principles/elements/patterns/strategies that account for this variable -->
- 

## Key Sources
-

Claim¶

---
type: claim
title: [Claim statement — one sentence, present tense]
id: [claim-slug]     # equal to the filename — what a design doc cites
status: draft
generated:
  by: <actor>
  at: YYYY-MM-DD
evidence_strength:   # strong / moderate / weak / mixed
---

# [Claim statement — one sentence, present tense]

> **Claim** · [All claims](index.md)

[Optional 1–2 sentence clarification of scope or mechanism.]

## Subclaims
<!-- Each subclaim is a 1-sentence lit-review summary.
     Prefix with quality and impact scores drawn from the supporting evidence.
     Link to the evidence anchor using standard markdown: [→ Author Year](#author-year) -->

`q? i?` [One-sentence summary of the finding and scope.] [→ Author Year](#author-year)

## Evidence
<!-- One entry per study. Heading slug becomes the anchor target for subclaim links,
     and (via scripts/migrate_to_okf.py's parser) the `id` of the matching entry in
     the frontmatter `sources:` list.
     Show full APA citation with DOI as a hyperlink.
     Then: quality · impact · n codes with plain-language explanations.
     Then: 2–4 sentence plain-language description. Link instructional elements to wiki pages.
     Avoid unexplained abbreviations or jargon. -->

### Author Year

Author, A., & Author, B. (Year). Title. *Journal, vol*(issue), pages. [doi:...](https://doi.org/...)

`q? · [e.g. peer-reviewed RCT / quasi-experiment / meta-analysis]` · `i? · [e.g. large effect, d=0.9]` · `n=?` · `[kind] · r?`

[2–4 sentences: study design, participants (who, how many, what context), conditions or intervention, and findings in plain language. Link any instructional elements used to their wiki pages, for example `[worked examples](../elements/demonstration.md)` and `[practice tasks](../elements/practice.md)`.]

## Discussion
<!-- Prose section covering: contradictions, moderators, boundary conditions, open questions.
     Link to related claim pages where relevant. -->

## Related Claims
-

Evidence kind and rigour. Every entry's codes line ends with a span naming what sort of evidence the study is and how well it does what that kind can do, e.g. qualitative · r3. Kinds: causal, quant-synthesis, review, associational, qualitative, design, theoretical. Rigour 1–3 is judged against the kind's own criteria (eval/kind-rigor/RUBRIC.md), never another kind's, so a careful interview study is r3 and a confounded trial r1; r? means the text available could not show it, and an entry with no text of its study is always r?. Both are mirrored into sources[] as kind: and rigour:. scripts/code_kind_rigour.py writes them, from the article or abstract, never from the wiki's own paraphrase. q stays, and is now read as the design tier: where a study sits on the causal-design ladder, which answers "how good is this as evidence of a cause" and not "how well was this done".

Design tiers (q): | q | Criteria | |---|----------| | 4 | Pre-registered RCT or well-powered meta-analysis | | 3 | Peer-reviewed experiment (not pre-registered) or systematic review | | 2 | Quasi-experiment, observational with controls, or narrative review | | 1 | Case study, expert opinion, or theoretical argument |

The codes live in three places, from one source. evidence-scales.json at the bundle root is the definition — as data, said once. docs_hooks/page_metadata.py loads it to render the human-facing panel, and each claim's frontmatter sources[] entries carry the per-study q/i/n mirrored out of the body by scripts/sync_evidence_codes.py. So an agent reads a tier and its meaning without parsing prose, and the reader's tables cannot drift from the agent's, because there is only one copy.

sources:
  - id: haak-et-al-2011
    resource: "https://doi.org/10.1126/science.1204820"
    title: "Haak, D. C., … (2011). Increased structure and active learning…"
    q: 3
    i: 2
    n: large (multiple course sections at a research university)
    kind: causal
    rigour: 2

Subclaim prefixes are deliberately not mirrored: a subclaim's q3 i2 is a reading of the evidence entry it links to, and that entry already carries the codes — a second copy would be a second place for them to disagree with no way to tell which was right. q? is preserved rather than dropped, because "somebody looked and could not establish it" is a different statement from an absent field.

These two tables are also rendered on every claim page, as a collapsed "Reading the evidence codes" panel inserted directly under ## Subclaims — see docs_hooks/page_metadata.py. An evidence entry spells its codes out beside them (q3 · quasi-experimental study · i2 · large effect · n=large), but a subclaim prefix is bare q3 i2, and across the claims corpus there are 465 quality codes and 443 impact codes with nothing next to them saying what the numbers mean. The tables were always here and this file is in the nav as Schema & Guide, so it was a proximity problem, not a missing docs one. Not tooltips: the codes sit inside code spans, and neither abbr nor Material's tooltips reach inside a <code> element.

A claim's header states its weight. scripts/add_evidence_summary.py puts a second banner line directly under the type banner, so an agent resolving research:<claim-slug> and a reader landing on the page both see what the claim rests on without scrolling:

> **Claim** · [All claims](index.md)
> **Evidence** · 1 study · `q3` peer-reviewed experiment · `i2` medium · n=48

It reads the same sources[] codes, so the header cannot disagree with the entries it summarises. It never collapses them into a verdict — a claim resting on studies coded q2 and q4 reports q2–q4, not "moderate". That is what evidence_strength: tried and why it produced nineteen spellings; the range is derived, a verdict would be invented. And a claim with no coded evidence says > **Evidence** · none recorded yet rather than carrying no line: 313 of 422 claim pages are stubs, and "no line" cannot be told from "the script never ran here".

The codes line is written two ways, and the parser must read both.

`q3 · peer-reviewed experiment · i2 · medium effect · n=48`        one span
`q3 · peer-reviewed experiment` · `i2 · medium effect` · `n=48`    three spans

The three-span form is dominant — 108 entries against 22 — and the first version of parse_evidence_codes required a single span, so it captured q from both and i/n from only the 22, silently. It had been written from one example page without checking which spelling the corpus used. Fixed: it takes every backtick span on the codes line and looks for each code independently, recovering i and n on 121 entries. Coverage went from 143/22/22 to 149/149/143.

Impact magnitude (i): until 2026-09-24 i0 meant "negligible or unclear", and the extraction prompt wrote i0 whenever an article printed no effect size. Prompt v128 and the validator now write null for that case, rendered i?. An older i0 may still mean either, so read its evidence text; the 11 claim pages coding i: 0 have not been recoded. | i | Rough effect size | |---|-------------------| | 3 | Large (d ≥ 0.8 or equivalent) | | 2 | Medium (d 0.4–0.79) | | 1 | Small (d 0.2–0.39) | | 0 | Negligible: an effect size is reported and it is below d 0.2 | | ? | No effect size reported: someone looked and the source does not say. Never read as 0 |

Ingest notes for agents¶

When a CSV field lists multiple items separated by commas or semicolons, expand each into a list item
When a field references another page by name (e.g., "Cognitive Load Theory"), convert it to a markdown link using the slugified name: [Cognitive Load Theory](../theories/cognitive-load-theory.md)
When research support / impact fields contain citations, extract them into ## Key Sources (and, once parsed, the frontmatter sources: list) and create or link sources/ pages
Mark pages status: review on ingest (no page stays draft since 2026-10-15); only a person's verified: entry makes a page stable
Never delete content on update — move superseded content to a  comment block
Never write a raw, un-slugified name straight into a file path — a name containing / (e.g. "Stand Up / Sit Down") will be interpreted as a subdirectory separator. Always pass names through slugify() before joining them into a path.