Calibration training and moderation to ensure reliable rubric-based portfolio review¶

Strategy · All strategies
Evidence · no claims cited

Description¶

A reader-capacity strategy in which the admissions director hires, trains, and coordinates 20 to 25 short-term contractual professionals who apply rubrics to applications. Per the report, "Prior to the application reading cycle, all Ross application readers undergo calibration training. During the cycle, the Ross Admissions Review Committee moderates scoring to ensure the rubrics are reliably applied for fair evaluations." Annual calibration trainings, monthly meetings, and ongoing moderation reports support confidence in rating reliability.

Design Implications¶

Context¶

Requirements¶

• Rubrics with clear criteria and detailed performance indicators
• A director to hire, train, and coordinate the reader team
• Annual calibration trainings, monthly meetings, and ongoing moderation reports

Constraints¶

• Readers exercise the same skill set as essay readers and do not require subject matter expertise

Target Learners¶

• Undergraduate BBA applicants evaluated through holistic review

Target Learning Goals¶

• Fair, reliable evaluation of qualitative portfolio submissions

Related Strategies¶

• Four recommended practices for implementing an admissions portfolio system
• Portfolio Assessment
• Peer and Self-Assessment

Examples¶

-

Key Sources¶

• Willis, L., & Martinez, M. R. (2022). Authentic student work in college admissions: Lessons from the Ross School of Business. Learning Policy Institute. https://doi.org/10.54300/756.774