Chatbot outreach produced no spillover effects on other-course performance or medium-term effects on subsequent enrollment¶

Claim · All claims
Evidence · 1 study · causal r3 · q3
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? No strong evidence of positive or negative spillover to same-term GPA in other courses, and no effects on subsequent-term enrollment in the subject, for the overall sample or for women in economics. → Meyer 2026

Evidence¶

Meyer 2026¶

Meyer, Katharine, Lindsay C. Page, Catherine Mata, Eric N. Smith, B. Tyler Walsh, C. Lindsey Fifield, Michelle Tyson, Amy Eremionkhale, Michael Evans, Shelby Frost, and Eye Eoun Jung. (2026). Let's Chat: Leveraging Chatbot Outreach for Improved Course Performance (NBER Working Paper No. 35397). National Bureau of Economic Research. doi:10.3386/w35397. Also released as EdWorkingPaper 22-564, Annenberg Institute at Brown University: https://edworkingpapers.com/ai22-564

q3 · i? · causal · r3

ITT analysis of term GPA (including and excluding the focal course) and next-term subject enrollment from administrative records (Table 9). Outcomes examined for the overall sample and for women in economics specifically; none showed significant treatment effects.

"We do not find strong evidence of positive or negative spillover or medium-term effects for the overall sample. We also consider each outcome for women in economics, where we previously found the strongest within-course effects, and did not observe significant effects of treatment on spillovers or subsequent semester enrollment for that subgroup."

Discussion¶

Related Claims¶

• Chatbot outreach showed no statistically significant effect on credits enrolled, earned, accumulated, or on term and cumulative GPA — related
• Chatbot outreach showed suggestive effects on assignment completion in Microeconomics but no effects on assignments in Government — related
• Academic chatbot outreach increased the likelihood students earned an A or B by four percentage points across two large online undergraduate courses — related
• Chatbot outreach increased supplemental instruction tutoring attendance by about two percentage points, mediating a small share of the grade effect — related
• Course completion effects differed by subject: reduced DFW in Government and reduced dropping in Microeconomics — related
• Women assigned to the chatbot in Microeconomics earned final grades seven points higher than control women, with no treatment effects for men — related
• Chatbot treatment effects on final grades were statistically significant between the 35th and 75th quantiles of the course grade distribution and at the 95th — related