Med2Lab

Product or Programme · All products and programmes
Evidence · 6 claims (5 for, 1 mixed) · 2 studies (2 design), q2–q3 · 0 of 2 report an effect size · 6 claims rest on one study

Description

Med2Lab is a platform used in medical education to submit case vignettes, open-ended questions, rubrics, and deidentified student responses to GPT-4 for automated scoring and feedback.

Components

• Med2Lab platform for feeding vignettes, rubrics, and student responses to GPT-4 for automated OEQ scoring: Med2Lab is a platform through which case vignettes, questions, rubrics, and deidentified student responses were entered into GPT-4 for automated scoring of open-ended exam questions. The article states that "The Med2Lab system was specifically engineered to manage rubric integration and automate prompt workflows." For each student response, GPT-4 generated a score and an explanation of 3–4 sentences of feedback. The model used was OpenAI's GPT-4o-2024-05-13, accessed via OpenAI's cloud platform through the Med2Lab interface at ZSOM and via a localized HIPAA-compliant deployment at UCSF. (Olvet DM et al. (2026))

Claims

• GPT-4 scoring errors clustered on multiple possible answers, unrecognized vocabulary, and missed incorrect or missing answers, while faculty were overly generous graders [+W]
• GPT-4 generated detailed per-response feedback that was useful for identifying scoring errors and could support personalized student feedback [+W]
• GPT-4 achieved almost perfect inter-rater reliability with faculty after three iterations of rubric engineering on three of four medical school OEQ questions [+W]
• Most faculty–GPT-4 scoring discrepancies were only one-point differences, shrinking across iterations [+W]
• The most challenging holistic-rubric question reached only moderate inter-rater reliability (kw=0.54) after three iterations [+W]
• GPT-4's error-diagnosis accuracy varies by error type: near-perfect on correct answers, weakest on Wrong Field and multiple-error responses [~W]

Related Products and Programmes

-

Key Sources

• Olvet DM, Kruidering M, Fulton TB, Truong B, Endo K, Lucito R, Willey JM. (2026). Can Generative Artificial Intelligence Reliably Score Open-Ended Question Assessments in Undergraduate Medical Education? Medical Science Educator 36:1539–1552. https://doi.org/10.1007/s40670-026-02638-2

Page metadata

Field	Value
type	product
id	med2lab
title	Med2Lab
description	Med2Lab is a platform used in medical education to submit case vignettes, open-ended questions, rubrics, and deidentified student responses to GPT-4 for automated scoring and feedback.
status	review
product_kind	software
generated	by process:settle-candidates on 2026-10-11
trust	unverified — no one has confirmed this page's content

Frontmatter is the page's machine-readable record — see the Schema & Guide.