Adding goal-monitoring prompts to a scripted conversational agent's weekly goal-setting raised undergraduates' self-reported goal attainment over reflection-only and goal-setting-only conditions¶

Claim · All claims
Evidence · 1 study · causal r2 · q3 · n=84 students
Currency · newest study 2025 · checked against its sources 2026-10-30 (automated, 1 of 1 entry) · not reviewed by a person

Subclaims¶

q3 i? In a randomised trial with 84 undergraduates at one Dutch university, all four conditions set weekly goals with the same scripted, non-generative agent; the two conditions that added goal-monitoring prompts (Monitoring Only, Monitoring & Reflection) reported higher goal attainment than Reflection Only and goal-setting-only Control. Goal attainment was self-reported, and the pairwise contrasts carry p values only. → Martins van Jaarsveld 2025

Evidence¶

Martins van Jaarsveld 2025¶

Martins van Jaarsveld, G., Wong, J., Baars, M., Specht, M., & Paas, F. (2025). Enhancing goal attainment in higher education with a scripted conversational agent: Effects of monitoring and reflection support in digital learning. Computers and Education. https://doi.org/10.1016/j.compedu.2025.105441

q3 · randomised controlled trial · i? · pairwise contrasts report p values only · n=84 students · causal · r2

Randomised controlled trial with 84 bachelor students in social sciences at one Dutch university over five weeks. A scripted, finite-state conversational agent (pre-authored prompts and branching, with feedback selected from a fixed set according to each goal's quality score) delivered weekly goal setting to every participant; conditions differed in whether it added goal-monitoring prompts (Thursday/Friday), reflection prompts (Monday/Tuesday, before the next goal setting), both, or neither (Control). Goal attainment was a weekly self-report on a 5-point scale, not a performance measure. A mixed-design ANOVA found a main effect of condition (F(3, 80) = 10.728, p < .001, η2p = .287 for the four-condition omnibus test); Bonferroni-corrected comparisons showed Monitoring & Reflection and Monitoring Only above Reflection Only and Control, with no difference between the two monitoring conditions or between Reflection Only and Control. Because every condition used the agent, the study tests the added monitoring prompts, not the agent itself; it does not show learning gains (academic performance did not differ) or delivery at scale. The authors' discussion nonetheless reads the result as showing that "a simpler, non-generative agent can still produce meaningful learning gains."

Discussion¶

Related Claims¶

• Monitoring support delivered via a scripted conversational agent significantly improves goal attainment compared with reflection-only and control conditions — related
• A daily learning diary alone (Group D) did not produce statistically significant pre-post gains on any measured outcome in an online mathematics preparation course — related
• Self-regulated learning improves achievement — a broader claim this one bears on
• Evidence suggests over-reliance on generative AI may negatively affect students' self-regulated learning behaviors, potentially harming retention and transfer to novel contexts — related
• As cited second-hand in an AI-literacy framework paper: ChatGPT use improved short-term essay scores but triggered metacognitive laziness with no significant knowledge-transfer gains — related