Developers of the principal practice instruments provided only partial information about the instruments' reliability and validity¶

Claim · All claims
Evidence · 1 study · associational r1 · q2
Currency · newest study 2015 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? The developers of the principal practice instruments used by pilot districts provided partial information about reliability (consistency across raters and observations) and validity (accurate measurement of true principal performance). → Christine Ross 2015

Evidence¶

Christine Ross 2015¶

Christine Ross, Mariesa Herrmann, Megan Hague Angus. (2015). Measuring Principals' Effectiveness: Results from New Jersey's Principal Evaluation Pilot. Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Mid-Atlantic. https://ies.ed.gov/ncee/edlabs/regions/midatlantic

q2 · i? · associational · r1

Documentary review of instrument developers' evidence for the practice instruments used in New Jersey's pilot. The report finds only "partial information" about reliability, defined as consistency across raters and observations, and validity, defined as accurate measurement of true performance.

"The developers of the principal practice instruments used by pilot districts provided partial information about the instruments’ reliability (consistency across raters and observations) and validity (accurate measurement of true principal performance)."

Discussion¶

Related Claims¶

• Principal practice ratings varied across the score range in New Jersey's pilot year, but most principals received ratings of effective or highly effective — related
• Defining and quantifying reliability and measurement error is particularly challenging in epistemic games for three principal reasons — a broader claim this one bears on
• Organizing performance measures into outcomes, impacts, and processes provides rich, fair, and diagnostically useful information — related
• Instruments show great heterogeneity in measurement quality, with validity or reliability information lacking in many cases — a broader claim this one bears on
• Validation evidence across the 15 RCR measures is partial and heterogeneous, with no test–retest, criterion validity or factor-analytic evidence reported — related
• Principals rated teaching practice reliably at the low and middle ends of the rating scale, and most principals' severity matched trained observers — related
• Professional practice ratings, student surveys, and value-added measures each have the potential to differentiate teacher performance in Pittsburgh Public Schools — related
• Actual classroom observation practice often differs from best practice, with principal-teacher perception gaps on measures and observer qualifications — related