No significant difference in perceived learning across the three modes (p = .845), despite the unrestricted mode's nearly double objective learning delta¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Participants rated how much they had learned similarly across modes (Full M = 6.06, Socratic M = 6.11, Adaptive M = 5.75), with no significant difference (p = .845). → Alexandre Clin Deffarges 2026 q2 i? Perceived difficulty was higher in the constrained modes (Socratic vs. Full: p = .041, d = -0.74) but did not correlate with objective EEG engagement (r = .126, p = .388). → Alexandre Clin Deffarges 2026 (2)

Evidence¶

Alexandre Clin Deffarges 2026¶

Alexandre Clin Deffarges, Nataliya Kosmyna, and Pattie Maes. (2026). Socrates went Nuclear: Comparing Interaction Strategies for AI systems in a Learning Context using Brain Sensing. arXiv preprint arXiv:2609.00584. https://arxiv.org/abs/2609.00584 doi:10.48550/arxiv.2609.00584

q2 · i? · causal · r2

Post-experiment questionnaire ratings of perceived learning on a 0-10 scale across the three modes. The authors call this the most unexpected finding and suggest Full-mode participants may have been aware their results were driven by answer retrieval rather than deep understanding.

"A one-way ANOVA revealed no significant difference between groups (𝐹( 2, 47)= 0.169,𝑝=. 845). Participants in Full mode, who achieved nearly double the learning delta of the other two conditions, did not perceive themselves as having learned more."

Alexandre Clin Deffarges 2026 (2)¶

Alexandre Clin Deffarges, Nataliya Kosmyna, and Pattie Maes. (2026). Socrates went Nuclear: Comparing Interaction Strategies for AI systems in a Learning Context using Brain Sensing. arXiv preprint arXiv:2609.00584. https://arxiv.org/abs/2609.00584

q2 · i? · causal · r2

Correlational analysis of post-questionnaire ratings against EEG engagement. Perceived difficulty was significantly higher in Socratic than Full mode (t=-2.143, p=.041, d=-0.74), but showed "no correlation (𝑟=. 126,𝑝=. 388,𝑛= 49)" with mean EEG engagement during training, and did not correlate with perceived learning either (r=-.06).

"perceived difficulty and mean EEG engage- ment during training showed no correlation (𝑟=. 126,𝑝=. 388,𝑛= 49), with the same null pattern within each condition."

Discussion¶

Related Claims¶

• In an immediate post-test design, participants with an unrestricted chatbot achieved significantly higher learning deltas than participants with a Socratic chatbot (d = 0.80) or an EEG-adaptive exercise system (d = 0.85) — related
• Practicing with AI did not create an illusion of mastery: participants reported learning and skill levels similar to other conditions — related
• Students' perceptions of mastery, emotions, and satisfaction were similarly high across all three feedback conditions — related
• Socratic participants required nearly twice as many attempts to reach the 80% mastery threshold as Full participants (d = -2.01), and 58% of the exercises of Socratic participants who abandoned the dialogue mid-way never reached the threshold — related