State of the Evidence¶

Generated by scripts/build_evidence_report.py from the claim pages' evidence codes and the [±~][SMW] markers on the pages that cite them. Do not edit by hand.

What the wiki's evidence adds up to. Every number counts distinct studies, keyed by DOI (or by author-year where a study has none), because one source often yields many claims. Nothing here is a verdict: tiers are reported as ranges and counts, and the impact codes are never averaged. q, i, kind and rigour are defined in the Schema & Guide and in evidence-scales.json.

In short¶

The wiki holds 6,930 distinct studies behind 26,894 claims. What they add up to (the sections below have the detail):

Few studies can show a cause. 31% are experiments or systematic reviews (q3 or q4); only 2% are pre-registered trials or well-powered meta-analyses (q4). 56% are quasi-experiments, observational studies or narrative reviews, and 12% are case studies, opinion or theory.
Few print an effect size. 1,687 of 6,930 studies (24%) report one for the finding the wiki cites; the rest report a significance test, a description, or nothing that says how large an effect was.
Strong work is not only trials. 817 studies are rated r3, done well for their kind; 516 of them are qualitative, design, theoretical or review work, which the design tier alone would rank low.
Most findings stand alone. 98% of claims rest on one study; a finding reported once, in one setting, has not been shown to hold elsewhere.
Not all research reaches design. 72% of studies, and 44% of claims, are cited by a principle, pattern, element, strategy or design; the rest are recorded but not connected to a design decision, either because none has been written or because the finding does not bear on one.
Principles and patterns. Of 224 pages written in the conditional format, 106 have research testing their central relationship, at least in part, and 118 carry a labelled default design because nothing in the wiki tests it. Their citations of claims are 5% strong, 42% moderate and 54% weak, and 49% supporting, 46% conditional and 6% against.

These figures describe two things at once: what education research has produced, and what this wiki has collected so far. As inclusion is held to a higher standard the second will move; the first is why it has to.

The corpus¶


Claims	26,894
Evidence entries	32,166
Distinct studies	6,930
Claims resting on one study	26,361 (98%)
Claims with no coded evidence	0
Studies reporting an effect size	1,687 of 6,930 (24%)

Studies by kind and rigour (rigour is judged against the standard of the study's own kind, so an r3 interview study and an r3 trial are each strong for what they are; r? means the text available could not show it):

kind	r3	r2	r1	r?	all
causal	97	883	296	384	1,660
quant-synthesis	116	189	42	135	482
review	164	536	111	143	954
associational	88	1,028	413	289	1,818
qualitative	126	323	71	94	614
design	61	545	135	35	776
theoretical	165	331	67	63	626

Studies by design tier q (the causal-design ladder; where one study is coded differently on two claims, the tier most of its entries give):

q1 case / opinion / theory	q2 quasi-experiment, observational, narrative review	q3 experiment or systematic review	q4 pre-registered RCT or well-powered meta-analysis
863 (12%)	3,905 (56%)	2,008 (29%)	154 (2%)

Studies per claim: 0: 0, 1: 26,361, 2: 448, 3: 68, 4 or more: 17.

Most claims rest on one study because of how the wiki grows: an extraction turns one article into claims, so every new claim starts with exactly one source, and gains a second only when a later article is merged into it. The single-study share measures how much merging remains, as much as it measures the literature.

Load-bearing studies¶

The studies the most claims rest on. A study that many claims rest on alone is a single point of failure: a correction to it, or a misreading of it, reaches every one of them.

Study	q	Claims citing it	Resting on it alone
Karpicke, J. D. (2017). Retrieval-Based Learning: A Decade of Progress. Learning and Memory: A Comprehensive …	q2	34	31
ClassWide Peer Tutoring: WWC Intervention Report. (2007). What Works Clearinghouse, U.S. Department of …	q3	25	25
John Q. Easton, Stephen Ponisciak, Stuart Luppescu. (2008). From High School to the Future: The Pathway to …	q2	20	20
Financial Incentives for Teen Parents to Stay in School. (2006). What Works Clearinghouse Intervention …	q3	18	17
Frith Chris D. (2012). The role of metacognition in human social interactions. Philosophical Transactions of …	q2	17	17
Price, H., Burns, D., Loewe, S., Shields, P., Kaplan, J., & Lee, H. (2024). Long-term English learners in …	q2	15	15
Breit, M., Schneider, M., & Preckel, F. (2025). Mathematics achievement and learner characteristics: A …	q3	15	15
Arthur I. Gates. (1921). An Experimental and Statistical Study of Reading and Reading tests (Concluded). The …	q2	14	14
WWC Intervention Report: ALAS. (2006). What Works Clearinghouse, U.S. Department of Education. …	q3	14	14
Dominguez, X., Rood, E., Kamdar, D., Leones, T., & Huynh, K. (2021). Splash and Bubbles for Parents App: …	q2	14	14
No author line. (1920). New Publications. The Journal of Educational Psychology. https://eric.ed.gov/	q1	13	13
Philip M. Gleason. (2017). What's the Secret Ingredient? Searching for Policies and Practices That Make …	q2	13	13
Crompton, H., Burke, D., Nickel, C., Bozkurt, A., Miao, F., Sharples, M., Greene, J. A., Parsons, D., …	q2	13	13
What Works Clearinghouse, U.S. Department of Education. (2006). WWC Intervention Report: Too Good for Drugs …	q3	12	12
Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students' …	q2	12	8
Karen E. Smith and Seth D. Pollak. (2020). Early life stress and development: potential mechanisms for …	q2	12	12
Project Cal-Well: Building Sustainable School-Based Mental Health Systems. (2024). WestEd. …	q2	12	12
Merging Cognitive Science and Curriculum to Strengthen Middle School Math. (2014). WestEd R&D Alert, Vol. 15, …	q2	11	11
Clark Douglas B., Tanner-Smith Emily E., Killingsworth Stephen S. (2016). Digital Games, Design, and …	q3	11	11
Reddy, S., Labutov, I., Banerjee, S., & Joachims, T. (2016). Unbounded Human Learning: Optimal Scheduling for …	q1	11	11

Citation load against evidence base¶

The claims the rest of the wiki cites most, beside what they rest on. A claim cited from hundreds of pages on the strength of two studies is where one more study, or a replication, would matter most.

Claim	Cited from	Studies	q	Effect size reported
Chunking reduces working memory load by grouping information into fewer, more meaningful …	571 pages	3	q2–q3	0 of 3
Prompting learners to self-explain improves understanding and problem solving on …	472 pages	4	q2–q4	1 of 4
Feedback Most Effective At Task And Process Levels	393 pages	2	q4	2 of 2
Self-determination theory holds that autonomy support raises intrinsic motivation, and a …	379 pages	3	q1–q4	1 of 3
Cognitive Overload Degrades Learning	345 pages	2	q3–q4	1 of 2
Learners' task-specific self-efficacy beliefs predict whether they initiate, persist …	330 pages	2	q2	0 of 2
Self-regulated learning models treat self-monitoring as a core process through which …	309 pages	2	q3	0 of 2
Comparing cases side by side improves learning and transfer by a moderate average amount, …	245 pages	3	q3–q4	1 of 3
Instructional guidance that helps novices can become redundant or counterproductive as …	230 pages	5	q1–q3	0 of 5
Belonging Interventions Improve Outcomes	229 pages	2	q3	0 of 2
Active Learning Improves Exam Performance	219 pages	3	q2–q4	2 of 3
Assessment for learning improves achievement	212 pages	3	q2–q3	2 of 3
Spaced Practice Improves Retention	188 pages	5	q3–q4	2 of 5
Contingent scaffolding improves learning more than fixed or absent support.	180 pages	5	q2–q4	0 of 5
Automatic word recognition frees resources for comprehension	171 pages	2	q1–q2	0 of 2
Fading support promotes the transfer of responsibility from instructor to learner.	170 pages	2	q3	0 of 2
Advance organizers aided learning in a 1980 meta-analysis, but activating prior topic …	151 pages	3	q3	1 of 3
Learners invest more effort in tasks they perceive as interesting, personally important, …	128 pages	2	q2–q3	0 of 2
Process goals lead to better skill acquisition for novices than outcome goals.	106 pages	2	q3	0 of 2
Worked-example guidance becomes less effective as learner expertise increases.	98 pages	2	q3	0 of 2
In one 37-student classroom experiment, recall and retention differ across media …	90 pages	1	q3	0 of 1
Adding relevant pictures or graphics to text improves learning from it, with a small …	86 pages	3	q2–q4	1 of 3
Interesting but irrelevant details can impair comprehension and transfer, with effects …	86 pages	6	q2–q3	1 of 6
In school studies, cooperative and peer-learning methods that reward groups on every …	82 pages	3	q2–q3	2 of 3
Building Empathy Improves Intergroup Attitudes	80 pages	3	q3–q4	0 of 3

Of the 39 claims cited from 50 or more pages, 6 rest on one study or none: In one 37-student classroom experiment, recall and retention differ …, Part-task practice reduces cognitive load for absolute novices during …, Relevancy of emphasized text directs attention and influences test …, A training-design argument, not tested by any study recorded here, …, Example-first conditions outperformed problem-first conditions on an …, Worked examples reduce unnecessary search for novices..

Contested claims¶

Claims cited as support on some pages and as counter-evidence on others ([+] and [-] markers on the same claim). The disagreement is in how pages use the claim, which is often the point: a boundary condition cited against a strategy it limits. It is still where a reader should look twice.

407 claims are cited both ways.

Claim	For	Mixed	Against	Studies
Cognitive Overload Degrades Learning	125	81	139	2
Interesting but irrelevant details can impair comprehension and transfer, with effects …	21	16	49	6
Self-determination theory holds that autonomy support raises intrinsic motivation, and a …	317	43	19	3
Minimal guidance is less effective for novices than explicit instruction	24	25	18	3
Chunking reduces working memory load by grouping information into fewer, more meaningful …	488	66	17	3
Learning Styles Matching Does Not Improve Learning	2	1	12	3
Coherence Principle Irrelevant Material Hurts Learning	22	5	11	4
Instructional guidance that helps novices can become redundant or counterproductive as …	9	211	10	5
Learners' task-specific self-efficacy beliefs predict whether they initiate, persist …	296	24	10	2
Praise for intelligence, rather than for effort, after success undermines children's …	3	7	8	3
Example-based sequences outperform problem-only practice for novices, and fading support …	44	1	8	2
Feedback Most Effective At Task And Process Levels	379	9	5	2
Fluent Illusions Mislead Self Assessment	2	6	5	2
PS-I effects vary by grade level: negative for 2nd–5th graders and increasing with age	1	1	5	1
Expected tangible rewards reduce later free-choice engagement with a rewarded task, while …	10	10	5	5
Prompting learners to self-explain improves understanding and problem solving on …	442	25	5	4
Executive functions do not directly predict syntactic complexity or story content once …	2	0	4	1
Taking initial multiple-choice tests without feedback can lead students to later produce …	2	1	4	1
Redundant on-screen text impairs learning when it competes with a visualization, though …	6	8	4	1
Worked examples reduce unnecessary search for novices.	63	9	4	1

Evidence by kind of page¶

Each page that cites claims carries its own profile under its title. Summarised by kind:

Kind	Pages	Citing a claim	Median studies behind a citing page
principles	170	170	29
elements	1,057	782	2
patterns	54	54	30
strategies	6,733	3,572	3
processes	18	16	9.5
methods	37	22	1.5
theories	1,360	915	1
learner-variables	75	75	49
designs	1,022	723	1
products	1,919	1,428	1
research-methods	641	562	1

Toward pooled estimates¶

A pooled effect size needs effect sizes with their uncertainty, on one scale, for comparable contrasts and outcomes. Those are recorded in observations/, which holds 27 study records: 2 report a standardised mean difference directly and 6 more report a statistic convertible to one (a correlation, an odds ratio, a partial η²). Fewer than three studies are on that scale at all, so no pooled estimate is reported. The wiki's i codes are not a substitute: they are bins, and averaging them would state a precision nobody measured.