Precision-weighted shrinkage in MAIHDA yields reliable conservative estimates for small strata and helps solve the multiple-testing problem¶

Claim · All claims
Evidence · 1 study · review r2 · q2
Currency · newest study 2023 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Shrinkage gives reliable (albeit conservative) estimates for strata with small sample sizes and controls error inflation without reducing power to detect true differences. → Keller 2023

Evidence¶

Keller 2023¶

Keller, L., Lüdtke, O., Preckel, F., & Brunner, M. (2023). Educational Inequalities at the Intersection of Multiple Social Categories: An Introduction and Systematic Review of the Multilevel Analysis of Individual Heterogeneity and Discriminatory Accuracy (MAIHDA) Approach. Educational Psychology Review. https://doi.org/10.1007/s10648-023-09733-5

q2 · i? · review · r2

Methodological exposition of the shrinkage factor applied to strata-level residuals, which increases when between-strata variance is low, within-strata variance is high, or stratum sample size is small. The article states shrinkage yields "reliable (albeit conservative) estimates for strata with small sample sizes".

"One advantage is that multilevel models thereby provide reliable (albeit conservative) estimates for strata with small sample sizes (Evans, 2019b; Evans et al., 2018). The second advantage is that the shrinking helps to solve the problem of multiple testing"

Discussion¶

Related Claims¶

• MAIHDA is more scalable and parsimonious than the traditional regression approach with full interaction terms — related
• Empirical Bayes shrinkage could be used to address bias in proxy pre-test value-added estimates — related
• Empirical Bayes shrinkage increases the precision of value-added estimates and reduces their absolute value for teachers of hard-to-predict students — related
• Teachers of students with hard-to-predict achievement levels tend to have less precise value-added estimates than teachers of other students — related
• In the PISA 2018 German illustration, students were assigned to 40 intersectional strata with sample sizes ranging from 3 to 512 — a narrower finding that bears on this claim
• A systematic review identified 21 publications applying intersectional MAIHDA analysis, primarily in the health sciences — related
• The variance partition coefficient from the simple intersectional model quantifies how useful intersectional strata are for predicting educational outcomes — related