Statistical tests applied to exploratory analyses lack evidential impact because the denominator of tests becomes unspecified¶
Claim · All claims
Evidence · 1 study · theoreticalr3·q1
Currency · newest study 2021 · not yet checked against its sources · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q1 i? If hypotheses and analyses are not predesignated, the denominator of possible tests is undefinable, so null-hypothesis tests lose their informative value; exploratory tests carry no evidential weight. → Preregistration and Registered Reports 2021
Evidence¶
Preregistration and Registered Reports 2021¶
Preregistration and Registered Reports. (2021). https://osf.io/ya594
q1 · i? · theoretical · r3
Theoretical argument from de Groot (1956/2014), quoted by the article: long-term error control in frequentist statistics depends on a defined denominator of tests, which post hoc analysis destroys. The article adds Gelman and Loken's garden of forked paths as the same problem.
"One “is allowed” to apply statistical tests in exploratory research, just as long as one realizes that they do not have evidential impact."
Discussion¶
Related Claims¶
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
exploratory-tests-lack-evidential-impact |
title |
Statistical tests applied to exploratory analyses lack evidential impact because the denominator of tests becomes unspecified |
description |
Statistical tests applied to exploratory analyses lack evidential impact because the denominator of tests becomes unspecified |
status |
review |
evidence_strength |
weak |
newest_evidence |
2021 |
generated |
by process:wiki-ingest on 2026-10-22 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — preregistration-and-registered-reports-2021 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.