Learners design poor experiments: confirmation bias, inconclusive designs, and ineffective variable manipulation are common¶

Claim · All claims
Evidence · 1 study (2 entries) · review r3 · q2
Currency · newest study 1998 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2003: baselines and conditions change

Subclaims¶

q2 i? The percentage of effective experiments (manipulating variables relevant to the tested hypothesis) could be as low as 22%. → Ton de Jong 1998 q2 i? Students rarely ran experiments intended to eliminate hypotheses: one out of six cases in a sample of 179 subjects. → Ton de Jong 1998 (2)

Evidence¶

Ton de Jong 1998¶

Ton de Jong, Wouter R. van Joolingen. (1998). Scientific Discovery Learning with Computer Simulations of Conceptual Domains. Review of Educational Research. https://telearn.hal.science/hal-00190680v1

q2 · i? · review · r3

The review reports two studies using a hypothesis scratchpad with simulation environments, in which learners manipulated irrelevant variables; "the percentage of effective experiments could be as low as 22%".

"In two studies, Van Joolingen and De Jong (1991b; 1993) found that learners often designed experiments in which variables were manipulated which had nothing to do with the hypothesis they were testing. The percentage of effective experiments could be as low as 22%."

Ton de Jong 1998 (2)¶

Ton de Jong, Wouter R. van Joolingen. (1998). Scientific Discovery Learning with Computer Simulations of Conceptual Domains. Review of Educational Research. https://telearn.hal.science/hal-00190680v1

q2 · i? · review · r3

The review reports Quinn and Alessi's influenza epidemic simulation experiment, where students chose experiment purposes from alternatives before running; only "one out of six in a sample of 179 subjects" aimed at eliminating hypotheses.

"Quinn and Alessi (1994) found that only in a small number of cases (one out of six in a sample of 179 subjects) students conducted experiments with the intention of “eliminating” hypotheses."

Discussion¶

Related Claims¶

• A systematic review of bias-mitigation training names considering the opposite as a previously validated way to reduce judgment biases, without testing it itself — related
• Manipulating stimulus physical properties and base rates of candidate causes changes preschoolers' exploratory play — related