Human-in-the-loop evaluation of LLM assessment conversations for fairness, bias, and explainability¶

Strategy · All strategies
Evidence · 1 claim (1 for) · 1 study (1 theoretical), q1 · 0 of 1 report an effect size · 1 claim rests on one study

Description¶

The article recommends that LLM-generated assessment conversations be evaluated with human oversight because prompts cannot guarantee fair, unbiased behavior across constructs and students. Since LLMs may perform differently depending on available training material and may include bias and hallucinations, "Human-in-the-loop approaches are needed to evaluate fairness and bias issues." The authors also call for approaches to improving explainability, such as documentation of how a conversation addresses aspects of the construct, and for investigating comparability of conversations across students. Additional time and effort are still required to ensure conversations are appropriate, free of bias, and that scores reflect those assigned by human experts.

Design Implications¶

Context¶

Requirements¶

• Human experts to verify that LLM-produced scores reflect expert-assigned scores
• Documentation approaches supporting explainability of how conversations address construct aspects

Constraints¶

• The article notes this strategy addresses challenges that remain open; innovative solutions are expected but not yet demonstrated

Target Learners¶

• all students assessed with LLM-based conversations

Target Learning Goals¶

• valid and fair assessment of knowledge, skills, and other attributes

Claims¶

• AI systems can produce biased or inaccurate diagnostic outputs, requiring critical teacher evaluation before integration [+W]

Related Strategies¶

• Build infrastructure-level mechanisms to monitor, benchmark, and control LLM response behavior across model transitions in conversation-based assessment

Examples¶

-

Key Sources¶

• Diego Zapata-Rivera, Carol Forsyth, Edith Aurora Graf and Yang Jiang. (2024). Designing and Evaluating Evidence-Centered Design based Conversations for Assessment with LLMs. Leveraging Large Language Models for Next Generation Educational Technologies, CEUR Workshop Proceedings. https://ceur-ws.org