In random experiments, both the ANCOVA and the CHANGE approach estimate the instruction effect without bias¶

Claim · All claims
Evidence · 1 study · theoretical r3 · q2
Currency · newest study 2021 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Given successful randomization, instruction is independent of the outcome at T1 and of the change-score, so both approaches return unbiased instruction effects. → Köhler 2021

Evidence¶

Köhler 2021¶

Köhler, C., Hartig, J., & Naumann, A. (2021). Detecting Instruction Effects—Deciding Between Covariance Analytical and Change-Score Approach. Educational Psychology Review. https://doi.org/10.1007/s10648-020-09590-6

q2 · i? · theoretical · r3

Analytical deduction within the article's hypothetical data-generating framework for the random-experiment scenario (Fig. 3). The article states that with successful randomization, neither stable nor time-specific components are associated with instruction, so "the instruction effect can be estimated without bias under both approaches".

"In a random experiment, instruction is independent of the outcome at T1 and it is also independent of the change-score, and the instruction effect can be estimated without bias under both approaches."

Discussion¶

Related Claims¶

• In nonrandom studies where trait variables influence instruction, the CHANGE approach is unbiased when change is unrelated to stable skill differences, the ANCOVA overestimates, and neither is unbiased when change depends on the trait — related
• In nonrandom studies where both trait and state variables influence instruction, neither approach reports an unbiased instruction effect — related
• In nonrandom studies where state-specific variables influence instruction, neither the ANCOVA nor the CHANGE approach reports the correct instruction effect — related