Positive subject-specific effects of Medly in Physics, Chemistry and Biology, with no evidence the effect differed across subjects¶
Claim · All claims
Evidence · 1 study · causalr2·q3·i2medium
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q3 i2 Positive intention-to-treat effects were observed in all three subjects: Physics g = 0.31, Chemistry g = 0.32, Biology g = 0.52, with no evidence of a subject-by-treatment interaction. → Wayne Harrison 2026
Evidence¶
Wayne Harrison 2026¶
Wayne Harrison, Rahil Khowaja, Emma Dobson, Germaine Uwimpuhwe, and Steve Higgins. (2026). Evaluating AI Tutoring at the Speed of Innovation: Practitioner-Led Micro-Randomised Trials of an AI Tutoring Platform in GCSE Science. https://arxiv.org/abs/2609.14789 doi:10.48550/arxiv.2609.14789
q3 · i2 · causal · r2
Subject-specific intention-to-treat mixed-effects models from the same trial (post-test n: Physics 312, Chemistry 217, Biology 115). All three estimates were positive, e.g. "Biology g = 0.52 (95% CI 0.19 to 0.84)".
"Positive effects were observed in all three subject-specific analyses. Physics produced g = 0.31 (95% CI 0.11 to 0.51), Chemistry g = 0.32 (95% CI 0.06 to 0.58), and Biology g = 0.52 (95% CI 0.19 to 0.84)."
Discussion¶
Related Claims¶
- Allocation to Medly improved post-test GCSE science attainment relative to business-as-usual self-directed revision (Hedges' g = 0.33) — a broader claim this one bears on
- Greater Medly engagement was positively associated with post-test attainment, treated as exploratory rather than causal — related
- No clear evidence that Medly's effect differed by Pupil Premium disadvantage status — related
- LLM teaching effectiveness varies substantially across subject domains, strongest where knowledge points apply directly and weakest in physics and chemistry — related
- Concept-mapping effect sizes vary by science subject (physics/earth science g = 1.040, biology g = 0.671, chemistry g = 0.590) but not significantly — related
- Intervention effects did not differ between physics and English subjects — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
medly-positive-effects-all-three-science-subjects |
title |
Positive subject-specific effects of Medly in Physics, Chemistry and Biology, with no evidence the effect differed across subjects |
description |
Positive subject-specific effects of Medly in Physics, Chemistry and Biology, with no evidence the effect differed across subjects |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2026 |
generated |
by process:wiki-ingest on 2026-10-10 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — wayne-harrison-2026 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.