System usability was rated low in both conditions, with no significant difference between them¶
Claim · All claims
Evidence · 1 study · causalr2·q3·i2medium
Currency · newest study 2026 · checked against its sources 2026-10-17 (automated, 1 of 1 entry) · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly
In Rief et al. (2026), seventh graders practised algebra in the "Math with Matt" tutoring system. The Mindful version added an LLM-based mindful chat, in which the pedagogical agent Matt gave context-sensitive emotional support, and mindful (not only evaluative) feedback and hint messages; the Cognitive version gave cognitive support only.
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q3 i2 The Mindful version received higher but not statistically significant SUS ratings than the Cognitive version (d=-0.48). → Rief 2026
Evidence¶
Rief 2026¶
Rief, V., Hladký, M., Yoo, M., Heel, S., Sato, S., Nagashima, T. (2026). Beyond Problem Solving: Large Language Models for Emotional and Reflective Support in Mathematics Learning. https://arxiv.org/abs/2609.02611 doi:10.48550/arxiv.2609.02611
q3 · i2 · causal · r2
System Usability Scale ratings in the 42-student experiment: Mindful M=55.56 (D grade) versus Cognitive M=43.96 (F grade) on the curved scale; the difference was not statistically significant, with printed d=-0.48.
"Although the Mindful version received higher SUS ratings than the Cognitive version, the difference was not statistically significant (t(39.74)=-1.594,p=.119, d=-0.48)."
Discussion¶
Related Claims¶
- The system receives a good average usability rating (SUS 69.7, grade B) but with wide individual variation, which the authors take to suggest UI consistency issues — related
- The FACTRIA-aware chatbot received good usability ratings (SUS 70.84) and high usefulness ratings from most of the 11 stakeholders — related
- Adults rated the LLM-powered iCub storytelling system as usable, with a total SUS score of 76.70 indicating good product usability — related
- The Bloom classification UI shows low perceived workload and high usability in a 50-participant user study — related
- Special-education practitioners rated AdaptED Stories as strongly usable, with a mean SUS score of 86.8 corresponding to an A grade — related
- Students rated the usability of the AIvaluate assessment in the SUS 'good' range — related
- Learner acceptance of AI simulations is generally high, with System Usability Scale scores of 61.6 to 78.56 and 72%-93% willingness to recommend — reports the opposite
- Both systems were usable with no significant differences in technical difficulties, practicality, or reading of messages — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
low-usability-both-conditions |
title |
System usability was rated low in both conditions, with no significant difference between them |
description |
System usability was rated low in both conditions, with no significant difference between them |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2026 |
source_checked |
2026-10-17 |
recheck_by |
2029 |
generated |
by process:wiki-ingest on 2026-10-11 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — rief-2026 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.