QBERT-RR finds high-confidence one-sentence Wikipedia answers for 463 WTC questions¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? After filtering by a 0.795 cosine-similarity confidence threshold, 463 of the 8,600 WTC questions can be answered from Wikipedia Summary. → Zhaozhen Xu 2024

Evidence¶

Zhaozhen Xu 2024¶

Zhaozhen Xu, Amelia Howarth, Nicole Briggs, Nello Cristianini. (2024). Understanding Visitors' Curiosity in a Science Centre with Deep Question Processing Network. Int J Artif Intell Educ. https://doi.org/10.1007/s40593-023-00377-8

q2 · i? · design · r2

Open-domain open-book QA retrieval over roughly 21M Wikipedia Summary sentences, embedded with QBERT-RR and searched via approximate nearest neighbour. The article reports "there are 463 questions in the corpus that can find an answer within the Wikipedia Summary" above the confidence threshold.

"After filtering the high-confidence answers, there are 463 questions in the corpus that can find an answer within the Wikipedia Summary using QBERT-RR."

Discussion¶

Related Claims¶

• QBERT-RR transfers to the unseen WTC question corpus with 90.76% equivalent-question recognition accuracy at a 0.809 threshold — related
• QBERT-RR outperforms the E5 embedding model on question topic and equivalence tasks while E5 leads on answer retrieval — related
• Raising the answer-confidence threshold reduces the number of retained MCQs but substantially improves their quality — related
• Graph community detection over QBERT-RR embeddings yields 5,930 question communities, with 5,337 questions having no similar group — related
• WTC corpus questions concentrate in Science & Mathematics (55.66%) and over half are causal HOW/WHY questions — related