Benchmarks are sensitive to precision and publication bias, but subgroup ordering remains stable and one-per-primary-study quartiles are broadly similar to raw values¶
Claim · All claims
Evidence · 1 study · quant-synthesisr3·q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q3 i? Inverse-variance weighting lowers overall quartiles to 0.09, 0.22, 0.49, while the one-per-primary-study specification yields 0.23, 0.52, 0.91; relative subgroup ordering changes little under all specifications. → Mehmet Bicakci 2026
Evidence¶
Mehmet Bicakci 2026¶
Mehmet Bicakci, Heidrun Stoeger, Diana Wengler, Albert Ziegler. (2026). Recalibrating impact in STEM education: Empirical benchmarks for interpreting intervention effects. Educational Research Review 52. https://doi.org/10.1016/j.edurev.2026.100803
q2 · i? · quant-synthesis · r3
Robustness analyses (precision weighting and one-per-primary-study specification, Table 3) show that after weighting "the relative ordering of subgroup distributions changed little". One-per-primary-study quartiles were 0.23, 0.52, 0.91.
"the overall 25th, 50th, and 75th percentiles declined from 0.21, 0.48, and 0.86 to 0.14, 0.32, and 0.59 after publication-bias adjustment, and further to 0.09, 0.22, and 0.49 under inverse-variance weighting. Despite this attenuation, the relative ordering of subgroup distributions changed little"
Discussion¶
Related Claims¶
- Publication-bias adjustment via limit meta-analysis lowers the overall STEM benchmarks to 0.14, 0.32, and 0.59 — related
- Raw effect size quartiles for STEM education interventions overall are 0.21, 0.48, and 0.86 at the 25th, 50th, and 75th percentiles — related
- Desired outcomes show markedly larger benchmarks than undesired outcomes, the strongest contrast in the study — related
- STEM field benchmarks differ markedly: science shows the largest quartiles and mathematics the smallest — related
- Cognitive, metacognitive, and strategic learning supports show the largest intervention-category benchmarks, while assessment and contextual interventions show the smallest — related
- Bias-adjusted posterior-predictive benchmarks for STEM moderator effect sizes (0.18, 0.38, 0.64 at the 25th, 50th, 75th percentiles) fall below both Cohen's conventional thresholds and raw empirical quartiles — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
benchmark-robustness-subgroup-ordering-stable |
title |
Benchmarks are sensitive to precision and publication bias, but subgroup ordering remains stable and one-per-primary-study quartiles are broadly similar to raw values |
description |
Benchmarks are sensitive to precision and publication bias, but subgroup ordering remains stable and one-per-primary-study quartiles are broadly similar to raw values |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2026 |
generated |
by process:wiki-ingest on 2026-10-21 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — mehmet-bicakci-2026 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.