The study's invariance findings may be biased by excluding students who missed a test and by sample heterogeneity¶

Claim · All claims
Evidence · 1 study (2 entries) · associational r2 · q1
Currency · newest study 2013 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q1 i? Students who missed one test were excluded from the longitudinal sample, which the authors state might introduce sample bias. → Wang 2013 q1 i? Samples were heterogeneous with respect to gender and ethnicity, and the impact of this heterogeneity on model parameter estimates is unknown. → Wang 2013 (2)

Evidence¶

Wang 2013¶

Wang, S., Jiao, H. and Zhang, L. (2013) 'Validation of longitudinal achievement constructs of vertically scaled computerised adaptive tests: a multiple-indicator, latent-growth modelling approach', Int. J. Quantitative Research in Education, Vol. 1, No. 4, pp.383–407. https://www.nwea.org/research/publication/validation-of-longitudinal-achievement-constructs-of-vertically-scaled-computerised-adaptive-tests-a-multiple-indicator-latent-growth-modelling-approach/ doi:10.1504/ijqre.2013.058307

q1 · i? · associational · r2

The authors' own stated limitation in the discussion: complete five-wave participation was required, so "students who missed one test were excluded from the sample". No effect size is printed; this is an author interpretation, not a tested result.

"However, since the study was based on longitudinal data, students who missed one test were excluded from the sample, which might introduce sample bias."

Wang 2013 (2)¶

Wang, S., Jiao, H. and Zhang, L. (2013) 'Validation of longitudinal achievement constructs of vertically scaled computerised adaptive tests: a multiple-indicator, latent-growth modelling approach', Int. J. Quantitative Research in Education, Vol. 1, No. 4, pp.383–407. https://www.nwea.org/research/publication/validation-of-longitudinal-achievement-constructs-of-vertically-scaled-computerised-adaptive-tests-a-multiple-indicator-latent-growth-modelling-approach/. doi:10.1504/ijqre.2013.058307

q1 · i? · associational · r2

Stated limitation: the pooled state samples were heterogeneous by gender and ethnicity, and the article states the impact on model parameter estimates is unknown. A cited prior study found statistically significant different rates of change by gender. No effect size is printed.

"All samples used in this study are not homogeneous with respect to at least gender and ethnicity, and the impact of heterogeneity on model parameter estimates is unknown."

Discussion¶

Related Claims¶

• The dashboard's cohort view restricts results to students who consistently tested across selected terms, and those outcomes may be positively biased — related
• The authors conclude that MAP reading and mathematics CAT constructs are invariant across five time points in ten states (except Indiana mathematics), though their mathematics tables show strong-invariance CFI drops above 0.01 in most states — related
• Some bias may remain in nonexperimental estimates even when rich pre-intervention data are used — a broader claim this one bears on
• Teacher effects may be heterogeneous across students of differing aptitudes, affecting growth modeling and teacher evaluation — related
• Sample truncation based on at-risk status can induce collider bias that undermines internal as well as external validity — related
• Participant samples in AI trust research skew WEIRD: mostly Western, adult, university-educated participants, limiting external validity — related
• Establishing longitudinal factorial invariance is neither sufficient nor necessary for examining developmental change — related
• Sample selection bias may have led the study to overestimate language outcomes for adolescents with long-term CI use — related