Adding multiple design elements to the basic ITS structure promotes causal inference by limiting alternative interpretations, demonstrated with No Child Left Behind¶

Claim · All claims
Evidence · 1 study · causal r? · q1
Currency · newest study 2015 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q1 i? Adding multiple design elements to the basic ITS structure serves to promote causal inference by limiting alternative interpretations, as demonstrated using No Child Left Behind as an example. → Manyee Wong 2015

Evidence¶

Manyee Wong 2015¶

Manyee Wong, Thomas D. Cook, Peter M. Steiner. (2015). Adding Design Elements to Improve Time Series Designs: No Child Left Behind as an Example of Causal Pattern-Matching. Journal of Research on Educational Effectiveness, vol. 8, no. 2. https://www.mathematica.org/publications/adding-design-elements-to-improve-time-series-designs-no-child-left-behind-as-an-example-of-causal doi:10.1080/19345747.2013.878011

q1 · i? · causal · r?

The article's central methodological demonstration, using the No Child Left Behind policy as a worked example of how added design elements strengthen ITS-based causal inference by limiting alternative interpretations.

"Using No Child Left Behind as an example, we demonstrate how adding multiple design elements to the basic ITS structure serves to promote causal inference by limiting alternative interpretations."

Discussion¶

Learner Variables¶

• Inferencing — outcome: instruction changes it

Related Claims¶

• The RCT is considered the gold standard of research evidence because randomization ensures only the intervention could cause treatment-control outcome differences — the general claim this finding bears on, with the studies behind it
• Added ITS design elements collectively provide a strong causal warrant when predictions are complex, results cohere with them, and no alternative fits the same pattern — a broader claim this one bears on
• Ambiguous preintervention functional forms and other factors affecting posttest means and slopes are common in ITS applications — related
• The single-group ITS design's counterfactual rests on the assumption that the pretest functional form can be validly estimated and extrapolated into the postintervention period, an assumption the authors call problematic — related
• Answering history explanation questions often requires causal inferences because causal relationships are frequently left implicit in textbooks — related
• Causal inference always goes beyond the data, even in randomized experiments, because experimental manipulations may affect other variables that caused the effect — related