Use pedagogical knowledge benchmarks to guide model selection and fine-tuning for educational AI, with ethical guardrails and human-in-the-loop deployment¶

Strategy · All strategies
Evidence · no claims cited

Description¶

The authors recommend that EdTech product developers use their leaderboards as "a practical guide for model selection, enabling them to make informed decisions that balance pedagogical performance with the inference cost and latency constraints of their specific tools", and that model developers use the benchmarks to guide fine-tuning of pedagogical practice. They further call for research into responsible deployment, including "clearethicalguardrails", user training protocols, and human-in-the-loop systems that augment rather than replace teacher professionalism.

Design Implications¶

Context¶

Requirements¶

• Access to the benchmark leaderboards and inference cost data for candidate models; awareness of the application's cost, performance and deployment constraints

Constraints¶

• The benchmarks assess pedagogical knowledge only and cannot measure practical dimensions of teaching such as classroom management or the ability to inspire learners; MCQ formats do not evaluate generative or interactive capabilities

Target Learners¶

• education technology product developers
• large language model developers
• policy makers deploying AI in education

Target Learning Goals¶

• selecting and improving LLMs for pedagogically sound educational applications

Related Strategies¶

• Convene cross-functional generative AI task forces to guide ethical adoption decisions and procurement
• Build educational AI with pedagogical guardrails such as withholding direct solutions and embedded reflection steps
• Center co-design with teachers and students throughout AI tool R&D, using human-in-the-loop workflows as a primary risk mitigation
• Evaluate and mitigate sentiment bias across sensitive attributes before deploying LLM forum support
• Benchmark AI assistants on specialized educational tasks before classroom deployment
• Deploy fine-tuned compact local 8B LLMs as free classroom and home story generators with adult human oversight and safety filtering
• Deploy quantized adapted small models offline with a teacher in the loop rather than as autonomous graders
• Match LLM deployment role in language education to the model's assessed proficiency level
• Use domain-specific fine-tuning of open-source diffusion models to build trustworthy generative AI tools for specialized technical domains

Examples¶

-

Key Sources¶

• Lelièvre, M., Waldock, A., Liu, M., Valdes Aspillaga, N., Mackintosh, A., Ogando Portela, M. J., Lee, J., Atherton, P., Ince, R. A. A., & Garrod, O. G. B. (2025). Benchmarking the Pedagogical Knowledge of Large Language Models. https://arxiv.org/abs/2506.18710