Involve domain experts throughout the AI safety evaluation pipeline, especially for defining unsafe content¶

Strategy · All strategies
Evidence · no claims cited

Description¶

The article recommends that future evaluations actively involve domain experts such as educators throughout the evaluation process, rather than only reflecting expert views through published guidelines as this study did. The authors state "directly engaging domain experts would yield a more credible and robust framework" and that expert participation is especially critical for establishing precise definitions of unsafe content, where vagueness remains a key challenge.

Design Implications¶

Context¶

Requirements¶

• Participation of domain experts such as educators across the evaluation pipeline

Constraints¶

Target Learners¶

• children and adolescents using generative AI

Target Learning Goals¶

• accurate, credible evaluation of AI safety risks for young users

Related Strategies¶

• Center co-design with teachers and students throughout AI tool R&D, using human-in-the-loop workflows as a primary risk mitigation
• Integrate co-design and continuous feedback loops into AI evaluation processes
• Involve teachers as co-designers of AI-based intelligent systems, with agency to train, correct, interpret, and override system recommendations
• Use expert appropriateness assessment of a designed learning model as the basis for further system development and future outcome studies
• Include an explicit intended-impact target in NLP evaluation and use noise-robust designs rather than only point estimates
• Human-in-the-loop generative pipeline for synthetic classroom training data with domain-expert validation

Examples¶

-

Key Sources¶

• Haein Kong. (2026). Child Safety in Generative AI: An Expert-Guided and Incident-Grounded Evaluation Framework. HEAL@CHI, April 2026, Barcelona. https://arxiv.org/abs/2607.00395