NSMQ Riddles¶
Product or Programme · All products and programmes
Evidence · 5 claims (5 for) · 1 study (1 design),q2· 0 of 1 report an effect size · 5 claims rest on one study
Description¶
A benchmark dataset created by George Boateng and colleagues containing 1,840 scientific and mathematical riddles from Ghana's National Science and Maths Quiz for evaluating automated question-answering systems.
Components¶
Claims¶
- The best LLMs perform below the best student contestants in the real-time proxy riddle round [+W]
- LLMs struggle most with Chemistry riddles, with best EM accuracy at or below 75% versus above 90% in other subjects [+W]
- LLM performance drops from offline to real-time proxy evaluation when clues arrive incrementally [+W]
- In offline evaluation on NSMQ Riddles, closed LLMs generally outperform open models and higher reasoning settings improve performance [+W]
- Answering early on few clues correlates with worse real-time proxy performance [+W]
Related Products and Programmes¶
Key Sources¶
- George Boateng, Naafi Dasana Ibrahim, Samuel John, Philemon Badu, Patrick Agyeman-Budu, Jonathan Abrefah Mensah, Kevin Takyi Yeboah, William Edor, Andrew Kojo Mensah-Onumah, Nana Sam Yeboah, and Victor Kumbol. (2026). NSMQ Riddles: A Benchmark of Scientific and Mathematical Riddles for Quizzing Large Language Models. arXiv preprint. https://arxiv.org/abs/2605.07051
Page metadata
| Field | Value |
|---|---|
type |
product |
id |
nsmq-riddles |
title |
NSMQ Riddles |
description |
A benchmark dataset created by George Boateng and colleagues containing 1,840 scientific and mathematical riddles from Ghana's National Science and Maths Quiz for evaluating automated question-answering systems. |
status |
review |
product_kind |
dataset |
generated |
by process:settle-candidates on 2026-10-11 |
trust |
unverified — no one has confirmed this page's content |
Frontmatter is the page's machine-readable record — see the Schema & Guide.