Usability testing with educators and developers identified chatbot strengths for independent challenge-solving but limited feedback for low-participation learners¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2025 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person · recheck by 2028: tool and model accuracy changes quickly

Subclaims¶

q2 i? Usability test participants found the chatbot design enabled learners to address challenges independently, facilitated team mediation, and provided precise guidance, while limited feedback for learners with low participation was identified as a weakness. → Minji Kim 2025

Evidence¶

Minji Kim 2025¶

Minji Kim, Cheolil Lim. (2025). AI chatbot design principles to enhance the collective efficacy in collaborative learning. Asia Pacific Education Review. https://doi.org/10.1007/s12564-025-10080-z

q2 · i? · design · r2

Usability test with eight participants in two groups (chatbot designers and instructors of collaborative learning), using semi-structured interviews of about 30 minutes analyzed thematically. Participants reported the design "enabled learners to address challenges independently" and supported team mediation.

"The chatbot's capabilities were found to include enabling learners to address challenges independently, facilitating mediation within the team, and offering precise guidance to prevent unproductive time expenditure."

Discussion¶

Related Claims¶

• Students valued the independence, real-world relevance, hands-on work, and collaboration in challenge based lessons — related
• In a pilot study, the TigerGPT survey chatbot achieved a 75% usability rating and an 81% satisfaction score, with 50% of participants preferring it over traditional surveys — related
• Gateway math students benefit from a balance of social, active learning and independent practice, but faculty constraints limit active learning — related
• Users rate chatbots easy to use with clear, relevant, largely accurate responses, but the intrusive pop-up design is the largest annoyance — related
• Faculty testing of AI chatbots on real student homework found feedback quality uneven, prompting-dependent, and focused on surface features — related
• AI chatbots provide students detailed feedback, step-by-step homework solutions, and self-quizzing support — related
• AI chatbots support skills development in writing, problem-solving, and group discussion facilitation — related
• Students found using a chatbot to solve a photoelectric-effect problem and evaluating its answer insightful, and the authors propose group checking as a way to verify chatbot answers — related
• Students primarily benefit from AI chatbots in three key areas: homework and study assistance, personalized learning, and skills development — related
• AI-supported prototyping combined with educator feedback loops accelerated development and surfaced usability issues early in one K-2 literacy tool project — related
• Chatbot-based instruction enhanced secondary students' learning experience by enabling self-paced study with less stress, time savings, and maintained motivation — related