The ML-LVM yields higher true-positive rates for detecting moderation effects of judgment accuracy components than the PS method across most conditions¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? In simulation study 2 with moderation effects of ± 0.5, the ML-LVM showed consistent true-positive-rate advantages over the PS method across conditions and components except in the smallest condition. → Julian F. Lohmann 2025

Evidence¶

Julian F. Lohmann 2025¶

Julian F. Lohmann, Nils Machts, Jens Möller, Steffen Zitzmann. (2025). A More Comprehensive, More Reliable Multilevel Approach for Assessing and Modeling Teacher Judgment Accuracy Using Latent Variables. Educational Psychology Review. https://doi.org/10.1007/s10648-025-10029-z

q2 · i? · design · r2

Simulation study 2 used a 2 × 3 design (N Level-2 = 50/200; N Level-1 = 10/30/50) with 500 datasets per condition and a large moderation effect of ± 0.5, comparing ML-LVM and PS detection of moderation effects via 95% credible intervals versus lm-based p-values. The article reports "consistent performance advantages in terms of true-positive rates occurred for the ML-LVM approach".

"Comparing the PS method and the ML-LVM, consistent performance advantages in terms of true-positive rates occurred for the ML-LVM approach across conditions and components except in the first condition."

Discussion¶

Related Claims¶

• The ML-LVM yields lower parameter bias and higher coverage rates for regression weights predicting learning outcomes than the PS two-step method — related
• The ML-LVM provides more accurate individual-level estimates of judgment accuracy components than the person-score (PS) two-step method — related
• The PS method shows systematic overestimation of the differentiation component that exceeds the 10% bias threshold in small and medium level-1 samples — related