AI-ALOE technology deployments were associated with improved adult learner outcomes, including higher course grades and retention for Jill Watson and higher-quality written work for SMART, though VERA outcomes varied by course structure¶

Claim · All claims
Evidence · 2 studies · 2 qualitative r2 · q2
Currency · no dated study recorded · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Use of Jill Watson was associated with higher average course grades and improved retention compared to control sections. → deploy-outcomes-1 q2 i? Learners receiving SMART AI-supported concept feedback produced higher-quality written work. → deploy-outcomes-1 q2 i? VERA results suggested gains in the complexity of learners' mental models, though outcomes varied by course structure and instructional emphasis. → deploy-outcomes-2

Evidence¶

deploy-outcomes-1¶

Jennifer M. Reddig, Glen R. Smith Jr., Sanaz Ahmadzadeh Siyahrood, Wesley G. Morris, Yoojin Bae, Kaitlyn Crutcher, John Kos, Rahul K. Dass, Jinho Kim, Momin Naushad Siddiqui, Daniel Weitekamp, Ploy Thajchayapong, Sandeep Kakar, Alex Endert, Scott Crossley, Min Kyu Kim, Chris Dede, Ashok Goel, and Christopher J. MacLellan. 2026. Guidelines for Designing AI Technologies to Support Adult Learning. In Designing Interactive Systems Conference (DIS '26), Singapore, Singapore. ACM, New York, NY, USA. https://doi.org/10.1145/3800645.3813102

q2 · i? · qualitative · r?

The article's deployment summary section reports, citing its references, that "Use of Jill Watson was associated with higher average course grades and improved retention compared to control sections" and that SMART learners produced higher-quality written work. No effect sizes or test statistics are printed for either comparison.

"Use of Jill Watson was associated with higher average course grades and improved retention compared to control sections [39]. Deployments of SMART showed that learners receiving AI-supported concept feedback produced higher-quality written work [4]."

deploy-outcomes-2¶

Jennifer M. Reddig, Glen R. Smith Jr., Sanaz Ahmadzadeh Siyahrood, Wesley G. Morris, Yoojin Bae, Kaitlyn Crutcher, John Kos, Rahul K. Dass, Jinho Kim, Momin Naushad Siddiqui, Daniel Weitekamp, Ploy Thajchayapong, Sandeep Kakar, Alex Endert, Scott Crossley, Min Kyu Kim, Chris Dede, Ashok Goel, and Christopher J. MacLellan. 2026. Guidelines for Designing AI Technologies to Support Adult Learning. In Designing Interactive Systems Conference (DIS '26), Singapore, Singapore. ACM, New York, NY, USA. https://doi.org/10.1145/3800645.3813102

q2 · i? · qualitative · r2

The deployment summary reports, citing its reference, that VERA "suggested gains in the complexity of learners’ mental models", with the article's own qualifier that "outcomes varied by course structure and instructional emphasis". No effect size is printed.

"Lastly, results from VERA suggested gains in the complexity of learners’ mental models, though outcomes varied by course structure and instructional emphasis [47]."

Discussion¶

Related Claims¶

• SMART® effects on comprehension were not statistically significant on either subtest, though the domain-average effect size was substantively important — related
• Early warning and proactive support agents are associated with reported retention and grade improvements in institutional deployments — a broader claim this one bears on
• Universities already use incipient AI such as IBM Watson to provide student advice around the clock, reducing administrative staffing needs — related
• The Jill Watson AI teaching assistant met student expectations in a Georgia Tech online course without being detected as non-human — related
• A single configurable analytics codebase replicates prior research findings across three educational AI assistant domains — related