KITE evaluation dataset of 109 instructor-verified questions from an Introduction to AI course¶
Product or Programme · All products and programmes
Evidence · 2 claims (1 for, 1 mixed) · 2 studies (2 design),q2· 0 of 2 report an effect size · 2 claims rest on one study
Description¶
A dataset of 109 instructor-verified questions and reference answers drawn from an Introduction to AI course's lecture slides and textbook, created for evaluating educational question-answering systems.
Components¶
Claims¶
- KITE's non-procedural responses show strong retrieval grounding (faithfulness 0.85, context relevance 0.94) and answer similarity 0.76, with lower factual correctness 0.45 [+M]
- Research-question responses score higher Answer Relevancy (0.87) than follow-ups (0.72), while Faithfulness, Context Relevance and Response Groundedness are lower [~W]
Related Products and Programmes¶
-
Key Sources¶
- Mragisha Jain, Tirth Bhatt, Griffin Pitts, Aum Pandya, Peter Brusilovsky, Narges Norouzi, Arto Hellas, Juho Leinonen, Bita Akram. (2026). Retrieval-Augmented Tutoring for Algorithm Tracing and Problem-Solving in AI Education. Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026), co-located with ACL 2026. https://arxiv.org/abs/2605.12988
Page metadata
| Field | Value |
|---|---|
type |
product |
id |
kite-evaluation-dataset-of-109-instructor-verified-questions-from-an-introduction-to-ai-co |
title |
KITE evaluation dataset of 109 instructor-verified questions from an Introduction to AI course |
description |
A dataset of 109 instructor-verified questions and reference answers drawn from an Introduction to AI course's lecture slides and textbook, created for evaluating educational question-answering systems. |
status |
review |
product_kind |
dataset |
generated |
by process:settle-candidates on 2026-10-11 |
trust |
unverified — no one has confirmed this page's content |
Frontmatter is the page's machine-readable record — see the Schema & Guide.