Center co-design with teachers and students throughout AI tool R&D, using human-in-the-loop workflows as a primary risk mitigation¶

Strategy · All strategies
Evidence · no claims cited

Description¶

The report recommends that AI developers "embrace and center the expertise of teachers and students by prioritizing co-design throughout a tool's R&D processes." Across risk sections, respondents described human-in-the-loop testing where experts and users evaluate output quality, and co-design to center users' needs. The report notes human-in-the-loop workflows were "highlighted as one of the most meaningful risk mitigation strategies" and recommends developers work more closely with teachers and students to co-design solutions meeting real needs.

Design Implications¶

Context¶

Requirements¶

• Involving experts and users in evaluating output quality during development
• Engaging established partners and maintaining transparency and communication channels with users

Constraints¶

• The report notes that despite nearly every submission mentioning equity, rarely were submissions intentional about meaningful co-design processes

Target Learners¶

• PK-12 mathematics students in the United States, particularly students who are Black, Latino, and/or from low-income backgrounds
• Teachers and instructional coaches

Target Learning Goals¶

• Equitable mathematics teaching and learning
• Safe, ethical, and reliable use of AI-enabled tools

Related Strategies¶

• Ensure communication and co-development with school leaders throughout edtech process rollout
• Involve teachers as co-designers of AI-based intelligent systems, with agency to train, correct, interpret, and override system recommendations
• Engage users and community members in the design process to make learning tools more attractive, useful, and effective
• Use iterative design with short feedback loops and multiple user-feedback methods during product development
• AI system developers should ensure human-centric design, institutional leaders should stay updated on global AI regulations, and LA experts should engage policymakers through dialogue
• Integrate co-design and continuous feedback loops into AI evaluation processes
• AI researchers should partner with developers and disseminate findings accessibly to bridge research-practice silos
• Use pedagogical knowledge benchmarks to guide model selection and fine-tuning for educational AI, with ethical guardrails and human-in-the-loop deployment
• Involve domain experts throughout the AI safety evaluation pipeline, especially for defining unsafe content
• Human-in-the-loop validation workflow: hold all AI-generated teaching content in unreviewed status until a teacher manually approves it
• Human-in-the-loop generative pipeline for synthetic classroom training data with domain-expert validation
• Integrate GPT models into LA-supported feedback systems with an iterative instructor-student validation loop to improve reliability

Examples¶

-

Key Sources¶

• Noakes, S., Shell, A., Van Nostrand, P., Murillo, A., Ruiz, P., and Liberman, B. (2024, September). AI-Powered Innovations in Mathematics Teaching & Learning: Initial Findings. Digital Promise. https://doi.org/10.51388/20.500.12265/229