The ML-LVM yields lower parameter bias and higher coverage rates for regression weights predicting learning outcomes than the PS two-step method¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? In simulation study 3, regression weights relating judgment accuracy components to learning outcomes showed smaller bias for the ML-LVM in almost all conditions, and the PS method showed consistently smaller 95% confidence-interval coverage rates. → Julian F. Lohmann 2025

Evidence¶

Julian F. Lohmann 2025¶

Julian F. Lohmann, Nils Machts, Jens Möller, Steffen Zitzmann. (2025). A More Comprehensive, More Reliable Multilevel Approach for Assessing and Modeling Teacher Judgment Accuracy Using Latent Variables. Educational Psychology Review. https://doi.org/10.1007/s10648-025-10029-z

q2 · i? · design · r2

Simulation study 3 simulated three outcome variables each strongly predicted by one component (β = 0.5) under four sample-size conditions, comparing ML-LVM and PS regression estimates on bias, RMSE, and coverage. The article reports "the two-step PS method showed consistently smaller coverage rates" and that PS regression weights were strongly underestimated (bias consistently above ± 10% and negative).

"As assumed, the two-step PS method showed consistently smaller coverage rates indicating that the true values of the simulated regression weights were less frequently included in the model-implied 95% confidence intervals than for the ML-LVM."

Discussion¶

Related Claims¶

• The ML-LVM yields higher true-positive rates for detecting moderation effects of judgment accuracy components than the PS method across most conditions — related
• The ML-LVM provides more accurate individual-level estimates of judgment accuracy components than the person-score (PS) two-step method — related
• The PS method shows systematic overestimation of the differentiation component that exceeds the 10% bias threshold in small and medium level-1 samples — related
• Unilevel OLS regression and four-level multilevel analysis of the same content analysis data yield differing p-values, especially for parameters that are not highly significant. — related