Select representative reduced portfolios with quality-control review and geographic-bias limits for standard setting¶

Strategy · All strategies
Evidence · no claims cited

Description¶

When full portfolios cannot be reviewed, select one entry per content strand whose subset performance most closely matches the whole-portfolio score, minimizing the z-score deviate difference. The report describes how samples were rank-ordered and "progressively reviewed to ensure that the samples did indeed exemplify an increasing level of student performance from the bottom to the top of the overall range," with problematic samples replaced after independent review by two reviewers.

Design Implications¶

Context¶

Requirements¶

• Limits on portfolios per school to prevent geographic over-representation
• Independent review of samples by two qualified reviewers

Constraints¶

• Judges were uncomfortable with reduced portfolios because it was difficult for the work to show multiple settings

Target Learners¶

• Students with significant disabilities whose portfolios are being standard-set

Target Learning Goals¶

• Accurate representation of the full range of student performance in standard setting materials

Related Strategies¶

Examples¶

-

Key Sources¶

• Olson, B., Mead, R., & Payne, D. (2002). A report of a standard setting method for alternate assessments for students with significant disabilities (Synthesis Report 47). https://eric.ed.gov/?id=ED472306