Chatbot outreach showed suggestive effects on assignment completion in Microeconomics but no effects on assignments in Government¶

Claim · All claims
Evidence · 1 study · causal r3 · q3
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? Treatment-assigned Microeconomics students were five percentage points more likely to complete practice quizzes and earned practice quiz grades 3.59 points higher; no treatment differences in assignment completion or performance appeared in Government. → Meyer 2026

Evidence¶

Meyer 2026¶

Meyer, Katharine, Lindsay C. Page, Catherine Mata, Eric N. Smith, B. Tyler Walsh, C. Lindsey Fifield, Michelle Tyson, Amy Eremionkhale, Michael Evans, Shelby Frost, and Eye Eoun Jung. (2026). Let's Chat: Leveraging Chatbot Outreach for Improved Course Performance (NBER Working Paper No. 35397). National Bureau of Economic Research. doi:10.3386/w35397. Also released as EdWorkingPaper 22-564, Annenberg Institute at Brown University: https://edworkingpapers.com/ai22-564

q3 · i? · causal · r3

ITT analysis of course deliverables in Microeconomics (Table 6) and Government (Table 5). Practice quizzes were worth 15% of the final grade; participation and reading checks (40% of grade) were unaffected. The authors report no treatment differences in assignment completion in Government.

"treatment-assigned students are five percentage points more likely to complete the available practice quizzes and thus, earn practice quiz grades 3.59 points higher; these quizzes were worth 15% of their final grade). The Microeconomic chatbot outreach does not affect course participation or reading checks (cumulatively worth 40% of the final grade)"

Discussion¶

Related Claims¶

• Course completion effects differed by subject: reduced DFW in Government and reduced dropping in Microeconomics — related
• Women assigned to the chatbot in Microeconomics earned final grades seven points higher than control women, with no treatment effects for men — related
• Academic chatbot outreach increased the likelihood students earned an A or B by four percentage points across two large online undergraduate courses — related
• Chatbot outreach increased supplemental instruction tutoring attendance by about two percentage points, mediating a small share of the grade effect — related
• End-of-course survey respondents strongly recommended the chatbot, but attitudinal outcomes showed no consistent treatment effects and survey participation was differentially selective — related
• Chatbot treatment effects on final grades were statistically significant between the 35th and 75th quantiles of the course grade distribution and at the 95th — related
• Quiz-level first-attempt performance analysis across cohorts identifies persistently challenging topics in online biomedical science courses — related
• Chatbot outreach showed no statistically significant effect on credits enrolled, earned, accumulated, or on term and cumulative GPA — related
• Participating in peer assessment causes a small but significant improvement in short-term achievement on unit quizzes in a college statistics course — related
• Chatbot outreach produced no spillover effects on other-course performance or medium-term effects on subsequent enrollment — related
• A strategic resource-use exam playbook intervention raised final course grades in introductory statistics across two randomized cohorts, with a dosage effect for completing it twice — related