In nonrandom studies where state-specific variables influence instruction, neither the ANCOVA nor the CHANGE approach reports the correct instruction effect¶

Claim · All claims
Evidence · 1 study · theoretical r3 · q2
Currency · newest study 2021 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Under data-generating models in which time-variant variables affect both the outcome at T1 and the instruction variable, both approaches yield biased instruction effects. → Köhler 2021

Evidence¶

Köhler 2021¶

Köhler, C., Hartig, J., & Naumann, A. (2021). Detecting Instruction Effects—Deciding Between Covariance Analytical and Change-Score Approach. Educational Psychology Review. https://doi.org/10.1007/s10648-020-09590-6

q2 · i? · theoretical · r3

Analytical deduction illustrated with the Holzberger et al. (2013) study of reciprocal effects between teacher self-efficacy and instructional quality. The ANCOVA partials out too much variance, underestimating the effect; the CHANGE score contains inseparable state-at-T1 and δ variance, so "neither of the approaches reports the correct effect".

"Unfortunately, neither of the approaches reports the correct effect of teacher self-efficacy under the data-generating models in Fig. 5."

Discussion¶

Related Claims¶

• In random experiments, both the ANCOVA and the CHANGE approach estimate the instruction effect without bias — related
• In nonrandom studies where both trait and state variables influence instruction, neither approach reports an unbiased instruction effect — a broader claim this one bears on
• In nonrandom studies where trait variables influence instruction, the CHANGE approach is unbiased when change is unrelated to stable skill differences, the ANCOVA overestimates, and neither is unbiased when change depends on the trait — related