In nonrandom studies where both trait and state variables influence instruction, neither approach reports an unbiased instruction effect¶

Claim · All claims
Evidence · 1 study · theoretical r3 · q2
Currency · newest study 2021 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? When unobserved stable and time-specific influences both confound instruction, neither the ANCOVA nor the CHANGE approach can deal with the confounding. → Köhler 2021

Evidence¶

Köhler 2021¶

Köhler, C., Hartig, J., & Naumann, A. (2021). Detecting Instruction Effects—Deciding Between Covariance Analytical and Change-Score Approach. Educational Psychology Review. https://doi.org/10.1007/s10648-020-09590-6

q2 · i? · theoretical · r3

Analytical deduction illustrated with the Lazarides and Buchholz (2019) PISA 2009/2010 study on student-perceived teaching quality and achievement emotions, where outcome and instruction were both measured at the student level at T1. The article concludes "neither of the approaches report an unbiased instruction effect".

"When either of the possible underlying data-generating models in Fig. 6 hold, neither of the approaches report an unbiased instruction effect: Neither of the approaches can deal with the confounding due to unobserved stable and time-specific influences."

Discussion¶

Related Claims¶

• In random experiments, both the ANCOVA and the CHANGE approach estimate the instruction effect without bias — related
• In nonrandom studies where state-specific variables influence instruction, neither the ANCOVA nor the CHANGE approach reports the correct instruction effect — a narrower finding that bears on this claim
• In nonrandom studies where trait variables influence instruction, the CHANGE approach is unbiased when change is unrelated to stable skill differences, the ANCOVA overestimates, and neither is unbiased when change depends on the trait — a narrower finding that bears on this claim