Research-question responses score higher Answer Relevancy (0.87) than follow-ups (0.72), while Faithfulness, Context Relevance and Response Groundedness are lower¶

Claim · All claims
Evidence · 1 study (2 entries) · design r2 · q2
Currency · newest study 2026 · checked against its sources 2026-10-20 (automated, 2 of 2 entries) · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? Research-question responses achieve stronger Answer Relevancy than follow-up responses (mean 0.87 vs 0.72), but mean Faithfulness is 0.52, Context Relevancy 0.40, and Response Groundedness 0.43 across all 30 queries. → Dana Rezazadegan 2026 q2 i? RQ3 achieves strong Faithfulness but a Response Groundedness score of zero, and RQ5 achieves the highest Answer Relevancy but a Context Relevancy score of zero. → Dana Rezazadegan 2026 (2)

Evidence¶

Dana Rezazadegan 2026¶

Dana Rezazadegan, Atie Kia, Phongpadid Nandavong, Dominique Carlon, Jeremy Nguyen, Abhik Banerjee, James Marshall, Anthony McCosker, Yong-Bin Kang. (2026). DysLexLens: A Low-Resource LLM Framework for Analysing Dyslexic Learners' Insights from Online Forums. https://arxiv.org/abs/2606.27619 doi:10.48550/arxiv.2606.27619

q2 · i? · design · r2

RAGAS evaluation section comparing research-question and follow-up responses over the same 30-query test set. The authors suggest performance is better when questions include keywords closely aligned with the concept dictionary, and report the comparison with mean scores of 0.87 and 0.72.

"The research question responses achieve stronger Answer Relevancy than follow-up responses, with mean scores of 0.87 and 0.72, respectively."

Dana Rezazadegan 2026 (2)¶

Dana Rezazadegan, Atie Kia, Phongpadid Nandavong, Dominique Carlon, Jeremy Nguyen, Abhik Banerjee, James Marshall, Anthony McCosker, Yong-Bin Kang. (2026). DysLexLens: A Low-Resource LLM Framework for Analysing Dyslexic Learners' Insights from Online Forums. https://arxiv.org/abs/2606.27619

q2 · i? · design · r2

Same RAGAS evaluation reporting evidential-support metrics for all 30 query responses: Faithfulness 0.52, Context Relevancy 0.40, Response Groundedness 0.43, all lower than Answer Relevancy for research questions.

"Across all 30 queries’ responses, the mean Faithfulness score is 0.52, mean Context Relevancy is 0.40, and mean Response Groundedness is 0.43."

Discussion¶

Related Claims¶

• DysLexLens achieves a mean Answer Relevancy of 0.75 across 30 responses to research questions and follow-ups on dyslexia-AI Reddit data — related
• DysLexLens is moderately robust to paraphrased queries (Answer Relevancy 0.58) but sensitive to keyword-perturbed queries (0.34) — related
• Of 100 human-audited claims, 39 are fully verifiable, 55 partially verifiable, and 6 not verifiable, with main responses showing stronger provenance than follow-ups — related
• Generated answers sometimes added correct background knowledge not explicitly supported by retrieved evidence, an unsupported-answer behavior the authors flag as requiring faithfulness evaluation — related
• The three specialized chatbot modes achieved their distinct pedagogical goals on RAGAS metrics: Quick Mode highest faithfulness (0.92), Deep Understanding near-perfect context recall (0.98) with intentionally low answer relevancy, Exam Coach highest answer relevancy (0.82) — related
• KITE's non-procedural responses show strong retrieval grounding (faithfulness 0.85, context relevance 0.94) and answer similarity 0.76, with lower factual correctness 0.45 — related
• The RAG-enhanced assistant achieves context recall above 0.93 and answer relevance of 0.87 on an expert-built 52-question evaluation set — related
• AlgoRAG answered all 179 TCS exam-style questions successfully with a mean response time of 38.0 seconds and a pedagogical quality score of 0.7620 — related
• Artifact-grounded agent responses are rated significantly higher than transcript-only responses on groundedness, analytical depth, helpfulness, relevance, and overall quality — related
• Diversity regularization λ = 0.30 was selected from a sensitivity analysis as balancing recommendation relevance and content diversity — related