The VCP rubric was difficult for teams to use because it lacked standardized calibration and its process and content categories were unclear¶

Claim · All claims
Evidence · 1 study · qualitative r? · q2
Currency · newest study 2019 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person · overdue for a recheck since 2022: tool and model accuracy changes quickly

Subclaims¶

q2 i? Rubric scores were challenging to interpret over time or across districts because the VCP rubric lacked a standardized calibration and scoring system, and the distinction between process and content was unclear to many district leaders. → Perry 2019

Evidence¶

Perry 2019¶

Perry, R., Marple, S., & Reade, F. (2019). What accelerates a community of practice? Inflection points that changed the course of the Math in Common Initiative. San Francisco, CA: WestEd. https://www.wested.org/project/math-in-common-evaluation/

q2 · i? · qualitative · r?

Qualitative evaluation of the VCP rubric used for developmental purposes only from 2014-15 onward. An Ed Partners staff member reflected that VCP "worked for a while, but we weren't getting people focused on improvement."

"One challenge was that the rubric lacked a standardized calibration and scoring system, making interpretation of scores over time, or across districts, difficult."

Discussion¶

Related Claims¶

• Rubrics with parallel sentence structure, unambiguous wording and well-specified dimensions have lower grading variance, and revising rubrics decreased median grading error from 12.4% to 9.9% — the general claim this finding bears on, with the studies behind it

• Criterion-referenced tests require empirically-based construct validation studies — the general claim this finding bears on, with the studies behind it

• Nevada's refined instructional materials review process addresses prior problems of unclear, non-standard, and inequitable review, including rural districts' obstacles to participation — related
• Districts face validity, reliability, comparability, and accountability-tension challenges in assessing future-ready skills — a broader claim this one bears on
• Integrating POG assessments into traditional standardized-test-centered accountability systems is complex, and lack of a standardized framework can produce inconsistent measurement across regions — related
• Using AI to evaluate AI outputs was often challenging because AI act as yes people and humans struggled to agree on evaluation rubrics — related
• District teams struggled to produce rigorous, reliable evidence for assessing their improvement work — related
• Case-study states rely on locally developed performance assessments created and piloted by teachers, with collaborative scoring — related
• Districts defined pilot success differently, making it difficult to set a standard for evaluating product effects on student improvement — related
• Global-judgment criteria of counselor effectiveness lack clarity, making replication fruitless and the research unproductive — related