Skip to content

Precision-based sample-size planning for clustered education trials

Research Method · All research methods
Evidence · 4 claims (4 for) · 4 studies (4 theoretical), q2 · 0 of 4 report an effect size · 3 claims rest on one study

Description

The article recommends that evaluators discuss "appropriate precision standards" and then determine, for each random-assignment design, "the required number of schools to achieve those standards" using empirical parameter values. This makes precision targets, rather than convenience, drive sample planning in education experiments. The principle applies the article's unified framework to practical evaluation design.

Accounts

  • Set precision standards for education trials before determining required school samples: The article recommends that evaluators discuss "appropriate precision standards" and then determine, for each random-assignment design, "the required number of schools to achieve those standards" using empirical parameter values. This makes precision targets, rather than convenience, drive sample planning in education experiments. The principle applies the article's unified framework to practical evaluation design. (Peter Z. Schochet (2008))
  • Plan evaluation sample sizes around the larger requirements of clustered RD designs before choosing between RD and experimental designs: Because clustered RD designs typically demand substantially larger samples than clustered experiments for equivalent statistical precision, evaluators should assess power implications before committing to an RD design. The report finds that "three to four times larger samples are typically required under RD than experimental clustered designs to produce impacts with the same level of statistical precision." This principle directs researchers to weigh that power cost against RD's applicability when the assignment point, pretests, and research questions make RD feasible. (Peter Z. Schochet (2008))
  • Size RCT data collection for teacher practice mediators according to the power needed for mediator-achievement analyses: Because teacher practice mediators are costly to obtain through classroom observations, videotaping, and coded protocols, the report argues power results "may have design implications for the scope of the data collection effort for obtaining costly teacher practice mediators." If power is low, mediator analyses reduce to a heuristic qualitative linking of outcomes. Designers should therefore decide the scope of mediator data collection in light of the school sample needed for precise mediator-achievement estimates. (Schochet (2009))
  • Empirical parameter inputs (intraclass correlations and regression R2 values) for education trial power calculations: The article assembles empirical values of "intraclass correlations, regression R2 values, and other parameters" as inputs for computing the required number of schools under each random-assignment design. These parameter values, drawn for standardized test scores of elementary school students, ground the article's power calculations in real data rather than assumptions. They are used to derive design-specific school sample requirements. (Peter Z. Schochet (2008))
  • Causal inference and HLM modeling framework for statistical power under clustered regression discontinuity designs: The report grounds its power analysis in a framework combining causal inference and hierarchical linear modeling (HLM) for clustered RD designs in education research. The theory is "grounded in the causal inference and HLM modeling literature", and the empirical work targets "commonly used designs in education research to test intervention effects on student test scores." Within this framework, clustering and the RD cutoff structure jointly determine the statistical precision of impact estimates, yielding the report's central finding that RD requires larger samples than randomized clustered designs. (Peter Z. Schochet (2008))
  • Statistical power formulas for linking teacher practice mediators to student outcomes in clustered school-based RCTs: The article develops statistical power formulas for exploratory analyses that estimate associations between student achievement outcomes and mediating teacher practice outcomes in randomized controlled trials of education interventions. The formulas are derived "under clustered school-based RCTs using ordinary least squares and instrumental variable estimators" and are then applied in a simulated power analysis. The framework serves researchers designing RCTs who want to test whether data support a study's conceptual model. (Peter Z. Schochet (2011))
  • Unified analytic framework for statistical power across school, classroom, and student random assignment designs: The article organizes power analysis for education experiments through a unified analytic framework that applies statistical methods from the literature across designs where "random assignment is conducted at the school, classroom, or student level". The framework lets the author compute required school samples for each design using "empirical values of intraclass correlations, regression R2 values, and other parameters". It serves as the organizing structure for the article's precision standards and design comparisons. (Peter Z. Schochet (2008))

Claims

Key Sources

  • Peter Z. Schochet. (2008). Statistical Power for Random Assignment Evaluations of Education Programs. Journal of Educational and Behavioral Statistics, vol. 33, no. 1. https://www.mathematica.org
  • Peter Z. Schochet. (2008). Technical Methods Report: Statistical Power for Regression Discontinuity Designs in Education Evaluations. Washington, DC: U.S. Department of Education, Institute of Education Sciences. https://ies.ed.gov
  • Schochet, Peter Z. (2009). Do Typical RCTs of Education Interventions Have Sufficient Statistical Power for Linking Impacts on Teacher Practice and Student Achievement Outcomes? (NCEE 2009-4065). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. http://ncee.ed.gov
  • Peter Z. Schochet. (2008). Statistical Power for Random Assignment Evaluations of Education Programs. Journal of Educational and Behavioral Statistics, vol. 33, no. 1. https://www.mathematica.org/publications/journal-article-statistical-power-for-random-assignment-evaluations-of-education-programs
  • Peter Z. Schochet. (2008). Technical Methods Report: Statistical Power for Regression Discontinuity Designs in Education Evaluations. Washington, DC: U.S. Department of Education, Institute of Education Sciences. https://www.mathematica.org/publications/technical-methods-report-statistical-power-for-regression-discontinuity-designs-in-education-evaluations
  • Peter Z. Schochet. (2011). Do Typical RCTs of Education Interventions Have Sufficient Statistical Power for Linking Impacts on Teacher Practice and Student Achievement Outcomes? Journal of Educational and Behavioral Statistics, vol. 36, no. 4. https://www.mathematica.org/publications/ja-do-typical-rcts-of-education-interventions-have-sufficient-statistical-power-for-linking-impacts
  • Peter Z. Schochet. (2005). Statistical Power for Random Assignment Evaluations of Education Programs. Princeton, NJ: Mathematica Policy Research. https://www.mathematica.org/publications/statistical-power-for-random-assignment-evaluations-of-education-programs
Page metadata
Field Value
type research-method
id precision-based-sample-size-planning-for-clustered-education-trials
title Precision-based sample-size planning for clustered education trials
description The article recommends that evaluators discuss "appropriate precision standards" and then determine, for each random-assignment design, "the required number of schools to achieve those standards" using empirical parameter values.
status review
moved_from element/empirical-icc-r2-inputs-education-power-calculations, theory/clustered-rd-power-framework-hlm, theory/power-formulas-mediator-linking-clustered-rcts, theory/unified-power-framework-multi-level-random-assignment
generated by process:settle-candidates on 2026-10-09
trust unverified — no one has confirmed this page's content
sources 3 — peter-z-schochet-2008, peter-z-schochet-2011, peter-z-schochet-2005

Frontmatter is the page's machine-readable record — see the Schema & Guide.