AlgoRAG answered all 179 TCS exam-style questions successfully with a mean response time of 38.0 seconds and a pedagogical quality score of 0.7620¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2026 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? AlgoRAG achieved a 100% success rate, 38.0 s mean response time, and pedagogical quality of 0.7620 across 179 questions, with low lexical-overlap scores. → Sushan Adhikari 2026

Evidence¶

Sushan Adhikari 2026¶

Sushan Adhikari. (2026). AlgoRAG: Retrieval-Augmented Generation for Theoretical Computer Science Education — A Comprehensive Evaluation Framework for Algorithm Analysis and Complexity Theory. arXiv preprint. https://arxiv.org/abs/2609.14572 doi:10.48550/arxiv.2609.14572

q2 · i? · design · r2

System evaluation on 179 expert-curated exam-style questions across seven TCS topics, each with instructor-written reference answers. The article reports "100% success rate" with 38.0 s mean response time, BLEU-4 of 0.0000, ROUGE-1 F1 of 0.0963, and pedagogical quality of 0.7620 (Table 1).

"AlgoRAG answered all 179 questions successfully (100% success rate) with a mean response time of 38.0 seconds. Table 1 summarizes the aggregate metrics."

Discussion¶

Related Claims¶

• Per-topic performance varied: NP-completeness scored highest ROUGE-1 F1 (0.1285), sorting algorithms highest pedagogical quality (0.8250), and recurrence relations lowest pedagogical quality (0.6629) — a narrower finding that bears on this claim
• BLEU-4 scored zero on all 179 mathematical-proof responses, which the authors interpret as a property of n-gram metrics rather than system failure — a narrower finding that bears on this claim
• Pedagogical-criterion breakdown shows worked examples in only about 42% of responses, the weakest criterion despite 312 practice problems in the knowledge base — a narrower finding that bears on this claim
• Research-question responses score higher Answer Relevancy (0.87) than follow-ups (0.72), while Faithfulness, Context Relevance and Response Groundedness are lower — related
• The three specialized chatbot modes achieved their distinct pedagogical goals on RAGAS metrics: Quick Mode highest faithfulness (0.92), Deep Understanding near-perfect context recall (0.98) with intentionally low answer relevancy, Exam Coach highest answer relevancy (0.82) — related