The ML-LVM provides more accurate individual-level estimates of judgment accuracy components than the person-score (PS) two-step method¶

Claim · All claims
Evidence · 1 study · theoretical r3 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? In simulation study 1, RMSE for individual-level judgment accuracy components was considerably higher for the PS method than for the ML-LVM across conditions, particularly at low level-1 sample sizes. → Julian F. Lohmann 2025

Evidence¶

Julian F. Lohmann 2025¶

Julian F. Lohmann, Nils Machts, Jens Möller, Steffen Zitzmann. (2025). A More Comprehensive, More Reliable Multilevel Approach for Assessing and Modeling Teacher Judgment Accuracy Using Latent Variables. Educational Psychology Review. https://doi.org/10.1007/s10648-025-10029-z

q2 · i? · theoretical · r3

Simulation study 1 compared the ML-LVM and PS methods on individual-level parameter estimates across a 3 × 3 design (N Level-2 = 50/100/200; N Level-1 = 5/10/30), with 250 datasets per condition. The article reports that "The RMSE was particularly high when the level-1 sample size was low" and differences remained evident at the highest sample size. No effect size is printed.

"The RMSE was considerably higher for the PS method across conditions and judgment accuracy components than for the ML-LVM. The RMSE was particularly high when the level-1 sample size was low."

Discussion¶

Related Claims¶

• The ML-LVM yields higher true-positive rates for detecting moderation effects of judgment accuracy components than the PS method across most conditions — related
• The ML-LVM yields lower parameter bias and higher coverage rates for regression weights predicting learning outcomes than the PS two-step method — related
• The PS method shows systematic overestimation of the differentiation component that exceeds the 10% bias threshold in small and medium level-1 samples — related