In an exploratory pilot (n=8), students rated LEA's usability, Chat Mode, Quiz Mode, learning outcomes, and trust mostly positively, with Chat Mode the strongest performer¶

Claim · All claims
Evidence · 1 study (2 entries) · design r2 · q2
Currency · newest study 2026 · checked against its sources 2026-10-20 (automated, 2 of 2 entries) · not reviewed by a person

Subclaims¶

q2 i? Pilot participants gave full agreement (8/8) on four of six usability items (7/8 on the Command Center item and on recommending LEA to peers) and on Chat Mode accuracy, comprehensibility, usefulness, and lecture complementarity. → Rumble 2026 q2 i? 8/8 agreed LEA helped understanding, confidence, and gap identification, and 8/8 trusted its accuracy, but other items were more mixed: 5/8 reported higher engagement, 3/8 endorsed the reversed additionality item, and 3/8 endorsed the reversed Chat Mode item "did not trust information". → Rumble 2026 (2)

Evidence¶

Rumble 2026¶

Rumble, T., Zarrin, J., Lovell, P. G., Falconer, R. (2026). Learning Engagement Assistant (LEA): Cross-Course Scalability and Classroom Evaluation of an Agentic AI Tutoring System. arXiv. https://arxiv.org/abs/2607.13370 doi:10.48550/arxiv.2607.13370

q2 · i? · design · r2

Exploratory pilot survey of eight CMP511 MSc students using a nine-section 7-point Likert instrument; no inferential statistics were reported. Table 4 shows 8/8 on four of six usability items and 7/8 on the Command Center and recommend-to-peers items (the prose says all usability items had 8/8). Full agreement (8/8) marked Chat Mode as "the strongest performer" on accuracy, comprehensibility, usefulness, and lecture complementarity.

"Chat Modewas the strongest performer: full agreement (8/8) on accuracy, com- prehensibility, usefulness, and lecture complementarity."

Rumble 2026 (2)¶

Rumble, T., Zarrin, J., Lovell, P. G., Falconer, R. (2026). Learning Engagement Assistant (LEA): Cross-Course Scalability and Classroom Evaluation of an Agentic AI Tutoring System. arXiv. https://arxiv.org/abs/2607.13370. doi:10.48550/arxiv.2607.13370

q2 · i? · design · r2

Descriptive survey results from the same eight-participant pilot: understanding, confidence and gap identification each drew 8/8, which the authors call "uniformly positive", while engagement uplift drew 5/8 and three respondents endorsed the reversed item that they would have learned as much without LEA. All eight trusted LEA's accuracy and were comfortable with data anonymisation, though 3/8 endorsed the reversed Chat Mode item "did not trust information"; the authors read these reversed-item endorsements as acquiescent responding, which n = 8 cannot settle.

"Learning outcomeswere uniformly positive on understanding (8/8), confidence (8/8), and gap identification (8/8)."

Discussion¶

Related Claims¶

• In simulation, LEA met or exceeded targets in Chat and Tutor modes while Quiz Mode's KC Alignment Score fell below target — related
• Tutor Mode's adaptive responsiveness was overestimated by simulation: only 3/8 pilot participants agreed LEA adapted appropriately when they expressed uncertainty — related
• A question-repetition bug in Quiz Mode, reported by 2/8 pilot participants, was a defect simulation-based evaluation could not identify — related
• Over 80% of pilot students agreed the AI proctor accurately understood their responses and scaffolded them appropriately — related
• Teachers rate LearnLens positively across usability, relevance, and accuracy, with all nine Likert items averaging at least 4.1/5 — related
• In an exploratory pilot, students perceived the course-specific RAG system's responses as highly aligned with course materials (89%) and helpful for assessment understanding — related
• Students rated peer tutoring with the VoicePod as acceptable and helpful for learning vocabulary — related