Evidence tracing links 96 of 114 generated claims to source chunks, but three claims with retrieval scores of 0.50 or below show retrieval similarity alone is insufficient¶

Claim · All claims
Evidence · 1 study (2 entries) · design r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? Of 114 unique claims across 30 responses, 96 link to at least one source-chunk identifier, and three claims with average retrieval scores of 0.50 or below show retrieval similarity alone does not guarantee claim-level evidential support. → Dana Rezazadegan 2026

Evidence¶

Dana Rezazadegan 2026¶

Dana Rezazadegan, Atie Kia, Phongpadid Nandavong, Dominique Carlon, Jeremy Nguyen, Abhik Banerjee, James Marshall, Anthony McCosker, Yong-Bin Kang. (2026). DysLexLens: A Low-Resource LLM Framework for Analysing Dyslexic Learners' Insights from Online Forums. https://arxiv.org/abs/2606.27619 doi:10.48550/arxiv.2606.27619

q2 · i? · design · r2

Evidence-tracing results across the 30 generated responses from the RAGAS evaluation, counting claims and their source-chunk links; 29 of 30 responses include at least one chunk identifier.

"At the sentence level, the export contained 114 unique claims, of which 96 are linked to at least one source-chunk identifier and 18 have no source-chunk identifier."

Dana Rezazadegan 2026 (2)¶

Dana Rezazadegan, Atie Kia, Phongpadid Nandavong, Dominique Carlon, Jeremy Nguyen, Abhik Banerjee, James Marshall, Anthony McCosker, Yong-Bin Kang. (2026). DysLexLens: A Low-Resource LLM Framework for Analysing Dyslexic Learners' Insights from Online Forums. https://arxiv.org/abs/2606.27619

q2 · i? · design · r2

Same evidence-tracing analysis noting that despite a mean retrieval score of 0.85 for identified claims, three claims score 0.50 or below, indicating a limit of similarity-based retrieval.

"However, three identified claims have an average retrieval score of 0.50 or below, showing that retrieval similarity alone is not sufficient for strong claim-level evidential support."

Discussion¶

Related Claims¶

• DysLexLens achieves a mean Answer Relevancy of 0.75 across 30 responses to research questions and follow-ups on dyslexia-AI Reddit data — related
• Standard semantic retrieval alone does not reliably recover the pedagogically required chunk types, with comparison chunks retrieved worst (14.3% on the real corpus) — related
• Retrieval Failure Reduces Benefit — related
• Retrieval practice improves long-term retention — related
• A git-history audit found 96.3% of AI-generated answer-key question blocks needed no post-release semantic repair — related
• NLI analysis reveals reference-based grading is limited: many semantically correct responses contradict reference answers — related
• KITE's non-procedural responses show strong retrieval grounding (faithfulness 0.85, context relevance 0.94) and answer similarity 0.76, with lower factual correctness 0.45 — related
• Retrieval failure explained only 2 of vector RAG's 11 ungrounded answers; 9 answers retrieved the correct material and were still ungrounded, suggesting chunked context invites fabrication — related