Allocation to Medly improved post-test GCSE science attainment relative to business-as-usual self-directed revision (Hedges' g = 0.33)¶
Claim · All claims
Evidence · 1 study · causalr2·q3·i1small
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q3 i1 In the primary intention-to-treat analysis, students allocated to Medly achieved higher post-test attainment than students undertaking business-as-usual self-directed revision, g = 0.33 (95% CI 0.18 to 0.48). → Wayne Harrison 2026
Evidence¶
Wayne Harrison 2026¶
Wayne Harrison, Rahil Khowaja, Emma Dobson, Germaine Uwimpuhwe, and Steve Higgins. (2026). Evaluating AI Tutoring at the Speed of Innovation: Practitioner-Led Micro-Randomised Trials of an AI Tutoring Platform in GCSE Science. https://arxiv.org/abs/2609.14789 doi:10.48550/arxiv.2609.14789
q3 · i1 · causal · r2
Multisite individually randomised controlled trial in English secondary schools; 929 students completed pre-test and 644 post-test. Adjusted mean post-test score was 13.95 in the intervention group and 11.39 in control, with "Hedges' g = 0.33 (95% CI 0.18 to 0.48)" in the combined primary intention-to-treat analysis.
"The adjusted coefficient was 2.56 marks (95% CI 1.39 to 3.73), corresponding to Hedges' g = 0.33 (95% CI 0.18 to 0.48)."
Discussion¶
Related Claims¶
- Greater Medly engagement was positively associated with post-test attainment, treated as exploratory rather than causal — related
- Implementation varied substantially across teacher trials, and business-as-usual revision was often already technology-rich — related
- Positive subject-specific effects of Medly in Physics, Chemistry and Biology, with no evidence the effect differed across subjects — a narrower finding that bears on this claim
- No clear evidence that Medly's effect differed by Pupil Premium disadvantage status — related
- Progressively larger treatment estimates among intervention students crossing engagement thresholds are hypothesis-generating only — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
medly-micro-rct-improves-gcse-science-attainment |
title |
Allocation to Medly improved post-test GCSE science attainment relative to business-as-usual self-directed revision (Hedges' g = 0.33) |
description |
Allocation to Medly improved post-test GCSE science attainment relative to business-as-usual self-directed revision (Hedges' g = 0.33) |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2026 |
generated |
by process:wiki-ingest on 2026-10-10 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — wayne-harrison-2026 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.