The FACTRIA-aware chatbot received good usability ratings (SUS 70.84) and high usefulness ratings from most of the 11 stakeholders¶

Claim · All claims
Evidence · 1 study (2 entries) · qualitative r3 · q2
Currency · newest study 2026 · checked against its sources 2026-10-24 (automated, 2 of 2 entries) · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? The overall SUS score was 70.84 ± 16.39, above the conventional threshold for good usability. → Marques 2026 q2 i? 10 of 11 participants rated the chatbot 4 or 5 out of 5 for providing meaningful support for contextualizing the data; confidence was moderate (3.7 ± 1.23) and varied. → Marques 2026 (2)

Evidence¶

Marques 2026¶

Marques, F., Ortiz-Beltrán, A., Amarasinghe, I., & Hernández-Leo, D. (2026). Responsible Institutional Analytics: Interpreting Bias with AI Support. Journal of Universal Computer Science, 33(6). https://arxiv.org/abs/2610.07205 doi:10.48550/arxiv.2610.07205

q2 · i? · qualitative · r3

Post-activity System Usability Scale questionnaire (Brooke et al. 1996) completed by the 11 study participants after the interpretation task. The score of "70.84 ± 16.39" is reported as above the conventional threshold for good usability.

"The overall SUS score was 70.84 ± 16.39, which is above the conventional threshold for good usability."

Marques 2026 (2)¶

Marques, F., Ortiz-Beltrán, A., Amarasinghe, I., & Hernández-Leo, D. (2026). Responsible Institutional Analytics: Interpreting Bias with AI Support. Journal of Universal Computer Science, 33(6). https://arxiv.org/abs/2610.07205. doi:10.48550/arxiv.2610.07205

q2 · i? · qualitative · r3

Post-survey questionnaire results from the think-aloud study. Participants reported moderate confidence ("3.7± 1.23"); P2 rated usefulness 2 out of 5 and P8 found prompts too generic, preferring more direct guidance.

"Responses to the post-activity questionnaire (see appendix V) indicated that most participants (10 out of 11) rated the chatbot highly (4 or 5 out of 5), for providing meaningful support for contextualizing the data."

Discussion¶

Related Claims¶

• Qualitative case narratives show the chatbot shifted stakeholders from narrow, visually driven readings toward conditional, context-aware interpretations — related
• A FACTRIA-aware chatbot increases the number of biasing sub-factors stakeholders consider when interpreting institutional analytics visualizations — related
• Special-education practitioners rated AdaptED Stories as strongly usable, with a mean SUS score of 86.8 corresponding to an A grade — related
• After design adjustments, the LAD achieved good–excellent usability (SUS M = 4.13, equal to 78.25%) — related
• Students rated the usability of the AIvaluate assessment in the SUS 'good' range — related
• The Bloom classification UI shows low perceived workload and high usability in a 50-participant user study — related
• Adults rated the LLM-powered iCub storytelling system as usable, with a total SUS score of 76.70 indicating good product usability — related
• System usability was rated low in both conditions, with no significant difference between them — related
• The system receives a good average usability rating (SUS 69.7, grade B) but with wide individual variation, which the authors take to suggest UI consistency issues — related
• In an 18-participant user study, the system received a 4.00/5 overall recommendation, with hallucination resistance rated 4.44/5 and highest ratings for ease of starting a conversation (4.39/5) and response relevance and accuracy (4.06/5) — related
• Learner acceptance of AI simulations is generally high, with System Usability Scale scores of 61.6 to 78.56 and 72%-93% willingness to recommend — related
• Participants held generally positive attitudes toward VISIGARP but rated diagram layout clarity low while finding layout easy to adjust — related
• Users rate chatbots easy to use with clear, relevant, largely accurate responses, but the intrusive pop-up design is the largest annoyance — related
• Students rated AISSA's usability as excellent, with an average SUS score of 83.38 from 30 of 46 pilot students — related
• Experimental-group participants rated TibetCPR highly usable (SUS = 84.3, an Excellent rating), with the highest items on voluntary engagement and independent use — related
• In a pilot study, the TigerGPT survey chatbot achieved a 75% usability rating and an 81% satisfaction score, with 50% of participants preferring it over traditional surveys — related