Field-tested IDSA prototype received moderate usability ratings: usage enjoyment highest, design and realization lowest¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2023 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? On a 5-point Likert scale, students rated usage enjoyment M = 3.75 (SD 1.79) and ease/intuitive use M = 3.44 (SD 1.65), with lowest ratings for design (M = 3.08, SD 1.58) and its realization (M = 2.93, SD 1.58). → Karrenbauer 2023

Evidence¶

Karrenbauer 2023¶

Karrenbauer, C., Brauner, T., König, C. M., & Breitner, M. H. (2023). Design, development, and evaluation of an individual digital study assistant for higher education students. Education Tech Research Dev. https://doi.org/10.1007/s11423-023-10255-8

q2 · i? · design · r2

Questionnaire evaluation within the field test, using six Likert-scale usability and user-interface items adapted from Bruner (2012); the first four questions drew 271 responses while the last two received only 19 answers and were excluded. The average feedback rating was 3.3 across categories, indicating "these characteristics require further improvement".

"The students particularly enjoyed the usage (M = 3.75, /u1D70E = 1.79) and per - ceived using the IDSA as relatively easy and intuitive (M = 3.44, /u1D70E = 1.65). Students gave their lowest ratings to the design (M = 3.08, /u1D70E = 1.58) and its realization (M = 2.93, /u1D70E = 1.58)"

Discussion¶

Related Claims¶

• Prototype functionalities moderately encouraged reflection on study goals, but perceived added value was limited and regular use was uncertain — related
• Most field-test students elected introductory tutorial information, supporting the tutorial requirement for usability — related
• Students reported positive usability and acceptance of MentorChat — related
• In an 18-participant user study, the system received a 4.00/5 overall recommendation, with hallucination resistance rated 4.44/5 and highest ratings for ease of starting a conversation (4.39/5) and response relevance and accuracy (4.06/5) — related
• Assignment support was the highest-rated Class Companion function (83% Effective/Very Effective, M = 3.38), while assessing knowledge and skills was the lowest (71%) — related
• Students rated Zoo U highly for engagement and likeability, with every usability area rated at least 3.8 on a four-point scale — related
• Students rate AI's helpfulness positively overall, with academic challenges receiving the highest usefulness, satisfaction, and intent-to-continue scores — related
• Stakeholder and literature analysis yields seven IDSA requirement groups covering functionalities, contact options, data-based individuality, testing, marketing, data protection, and usability — related