In the PIRLS 2006 teaching-quality example, the optimal fixed-budget design was 395 classes with 12 students per class, and the actual PIRLS design achieved 83% power¶

Claim · All claims
Evidence · 1 study (2 entries) · theoretical r3 · q2
Currency · newest study 2022 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Applying the fixed-budget procedure to the Arens and Morin (2016) parameter values yielded optimal numbers of 395 classes and 12 students per class, with 73% power to detect a standardized between slope of .1. → Steffen Zitzmann 2021 q2 i? The most cost-efficient design for 80% power required 469 classes and 12 students per class with a budget of 71,250€, and the actual PIRLS design had 83% power. → Steffen Zitzmann 2021 (2)

Evidence¶

Steffen Zitzmann 2021¶

Steffen Zitzmann, Wolfgang Wagner, Martin Hecht, Christoph Helm, Christian Fischer, Lisa Bardach, Richard Göllner. (2021). How Many Classes and Students Should Ideally be Sampled When Assessing the Role of Classroom Climate via Student Ratings on a Limited Budget? An Optimal Design Perspective. Educational Psychology Review (2022) 34:511–536. https://doi.org/10.1007/s10648-021-09635-4

q2 · i? · theoretical · r3

Illustrative application assuming a fixed budget of 60,000€, costs of 68.20€ per class and 7.05€ per student, ICC(1)s of .32 (achievement) and .12 (support ratings), a standardized within slope of 0.159, and a target between slope of .1. The optimal design had 19 classes and 6 students per class less than the actual PIRLS sample.

"Applying our procedure for determining the design with the maximum power given a fixed budget to these values provided optimal numbers of 395 classes and 12 students per class."

Steffen Zitzmann 2021 (2)¶

Steffen Zitzmann, Wolfgang Wagner, Martin Hecht, Christoph Helm, Christian Fischer, Lisa Bardach, Richard Göllner. (2021). How Many Classes and Students Should Ideally be Sampled When Assessing the Role of Classroom Climate via Student Ratings on a Limited Budget? An Optimal Design Perspective. Educational Psychology Review (2022) 34:511–536. https://doi.org/10.1007/s10648-021-09635-4

q2 · i? · theoretical · r3

Cost-minimization application with the same parameter values and a target power of 80%. Only the number of classes increased (by 74) relative to the fixed-budget optimum, and the article concludes the actual PIRLS design was sufficiently powered because "the power was 83%."

"we obtained an optimal design with 469 classes and 12 students using our procedure for finding the most cost-efficient design with a power of 80%. The required budget was 71,250€"

Discussion¶

Related Claims¶

• The variability of the between slope estimate in the multilevel latent covariate model decreases with more classes and students, with higher ICC(1) of the ratings and larger slopes, but increases with higher ICC(1) of the outcome — related
• Under a fixed budget, a higher ICC(1) of the outcome variable shifts the optimal design toward more classes and fewer students per class and reduces maximum power — related
• A higher ICC(1) of students' ratings and a larger standardized between slope increase power, with opposite effects on the optimal split between classes and students per class — a broader claim this one bears on