LLM accuracy on the CDPK pedagogy benchmark ranges from 28% to 89%, with closed-weight reasoning models dominating the top of the leaderboard¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person · recheck by 2028: tool and model accuracy changes quickly

Subclaims¶

q2 i? Across 97 tested LLMs, CDPK accuracy spans 28% (Llama-3.2 1B) to 89% (Gemini 2.5 Pro), and 9 of the top 10 models are closed-weight models from OpenAI, Google and Anthropic. → Lelièvre 2025

Evidence¶

Lelièvre 2025¶

Lelièvre, M., Waldock, A., Liu, M., Valdes Aspillaga, N., Mackintosh, A., Ogando Portela, M. J., Lee, J., Atherton, P., Ince, R. A. A., & Garrod, O. G. B. (2025). Benchmarking the Pedagogical Knowledge of Large Language Models. https://arxiv.org/abs/2506.18710 doi:10.48550/arxiv.2506.18710

q2 · i? · design · r2

Benchmark evaluation of 97 LLMs on the 899 tested CDPK questions using a fixed few-shot prompt and exact-match accuracy with 95% bootstrap confidence intervals. The article reports accuracies "from 28% (Llama-3.2 1B) to 89%" and that closed-source models dominate the top of the leaderboard.

"Accuracies range from 28% (Llama-3.2 1B) to 89% for the current leader, Google's Gemini 2.5 Pro model. Closed-source models dominate at the top of the leaderboard (Figure 6), with 9 out of the top 10 being closed models from OpenAI, Google and Anthropic."

Discussion¶

Related Claims¶

• CDPK performance scales with model size, with a sharp Pareto-frontier drop-off below around 8B parameters — related
• The benchmark procedure is stable under stochastic decoding, with a maximum accuracy standard deviation of 0.43% over 20 repeated runs — related
• Estimated human baseline on the CDPK benchmark is approximately 50%, based on aggregate Chilean teacher exam results — related
• Both pedagogy-expertise-weighted and unanimous-vote ensembles frequently worsen LLM alignment with student learning — related
• In the blind judicial-exam evaluation, Gemini 2.5 Pro scored 21/24, above the top human benchmark of 18/24, while the other LLMs scored substantially lower. — related
• The EduQwen 32B models achieve up to 96.52% accuracy on the CDPK Benchmark, surpassing the previous leaderboard leader Gemini-3 Pro at 90.55% — related
• Expert tutors' lesson-planning and practice-design leaderboards separated AI models, and expert-rated practice design correlated with fewer student answer disputes — related
• Over a quarter of benchmark tasks (23 of 80) defeat all six frontier models, with over 22% showing capability boundaries where scores swing by up to 90 points — related
• Accuracy-cost value frontier: at $0.10 per million tokens, CDPK performance rose from 50% (April 2024) to 82% (June 2025) — related
• Current GenAI tools are not designed to support active learning and their apparent domain expertise is brittle — related