Learning performance improved over time and differed by group, but ADA showed no significant group-by-time interaction advantage over conventional materials¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r2 · q3 · i3 large
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i3 RM-ANOVA on pre-post performance showed a significant main effect of time (ges = 0.22) and of group (ges = 0.03), but no significant group-by-time interaction (p = 0.094). → Cilia Ricarda Rücker 2025 q3 i? At subtask level, the application task A3 came closest to a significant interaction favoring the chatbot group (F = 3.18, p = 0.076). → Cilia Ricarda Rücker 2025 (2)

Evidence¶

Cilia Ricarda Rücker 2025¶

Cilia Ricarda Rücker, Sebastian Becker-Genschow. (2025). Enhancing enthusiasm for STEM education with AI: Domain-specific chatbot as personalized learning assistant. Computers and Education Open. https://doi.org/10.1016/j.caeo.2025.100315

q3 · i3 · causal · r2

Repeated-measures ANOVA on learning performance in the cluster-randomized study (N = 195), used because pre-test prior knowledge differed significantly between groups. The time effect had a large printed effect size "ges=0.22" and the group effect ges=0.03; despite the CG's higher pre-test (M = 8.61 vs EG 3.56), the EG improved more (post M = 12.42 vs 9.38), but the interaction was non-significant.

"An RM-ANOVA confirmed a significant main effect of time, F (1194)=145.76, p < 0.001, ges=0.22, and group, F(1194)=9.26, p = 0.003, ges=0.03, but no significant group by time interaction, F(1194)= 2.84, p = 0.094"

Cilia Ricarda Rücker 2025 (2)¶

Cilia Ricarda Rücker, Sebastian Becker-Genschow. (2025). Enhancing enthusiasm for STEM education with AI: Domain-specific chatbot as personalized learning assistant. Computers and Education Open. https://doi.org/10.1016/j.caeo.2025.100315

q3 · i? · causal · r2

Subtask-level RM-ANOVA in the same study: each subtask showed a significant time effect (all ps<0.001) and group differences for A1, A3, and A41, but no interaction reached significance; the application task A3 (EG M = 9.44 vs CG M = 6.68) was closest.

"Although no interaction reached significance at the 0.05 level, Task A3 came closest, F(1194)=3.18, p = 0.076"

Discussion¶

Related Claims¶

• ADA produces directional but non-significant trends toward more positive-activating and less negative-deactivating emotions versus conventional materials — related
• Students in secondary mathematics report high acceptance of the domain-specific AI chatbot ADA across all Technology Acceptance Model dimensions — related
• Using the AI chatbot ADA significantly enhances ninth-graders' situational interest in mathematics compared with conventional differentiation materials — related
• ADA use shows a modest non-significant tendency toward increased intrinsic and extrinsic cognitive load, contrary to prior chatbot research — related