DysLexLens is moderately robust to paraphrased queries (Answer Relevancy 0.58) but sensitive to keyword-perturbed queries (0.34)¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? Keyword-perturbed queries show a clear decrease in Answer Relevancy to 0.34 versus 0.75 for original queries, indicating greater sensitivity when domain-specific terms are changed. → Dana Rezazadegan 2026

Evidence¶

Dana Rezazadegan 2026¶

Dana Rezazadegan, Atie Kia, Phongpadid Nandavong, Dominique Carlon, Jeremy Nguyen, Abhik Banerjee, James Marshall, Anthony McCosker, Yong-Bin Kang. (2026). DysLexLens: A Low-Resource LLM Framework for Analysing Dyslexic Learners' Insights from Online Forums. https://arxiv.org/abs/2606.27619 doi:10.48550/arxiv.2606.27619

q2 · i? · design · r2

Query robustness analysis comparing original, paraphrased, and keyword-perturbed variants of all 30 questions under the same configuration (Table 1: original 0.75, paraphrased 0.58, keyword-perturbed 0.34 for Answer Relevancy).

"However, the keyword-perturbed queries show a clear decrease in Answer Relevancy (0.34), indicating that the framework is more sensitive when domain-specific terms are changed."

Discussion¶

Related Claims¶

• DysLexLens achieves a mean Answer Relevancy of 0.75 across 30 responses to research questions and follow-ups on dyslexia-AI Reddit data — related
• Research-question responses score higher Answer Relevancy (0.87) than follow-ups (0.72), while Faithfulness, Context Relevance and Response Groundedness are lower — related
• Model performance is robust to minor prompt wording changes but sensitive to holistic rubric redesign — related
• Small input perturbations such as minor spelling errors can reduce state-of-the-art dialogue and assessment model accuracy by over 30% — related
• Post-deployment diagnostics attributed many remaining retrieval misses to query formulation, with bounded expansion recovering eleven of 24 original queries without lowering the relevance floor
• General-purpose embedding yields better retrieval while domain-specific embedding yields higher answer completeness and relevance — related