Socratic participants required nearly twice as many attempts to reach the 80% mastery threshold as Full participants (d = -2.01), and 58% of the exercises of Socratic participants who abandoned the dialogue mid-way never reached the threshold¶

Claim · All claims
Evidence · 1 study · causal r2 · q2 · i3 large
Currency · newest study 2026 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person

Subclaims¶

q2 i3 Socratic participants required nearly twice as many attempts to reach the 80% threshold (M = 2.80) as Full participants (M = 1.46), a very large effect (d = -2.01). → Alexandre Clin Deffarges 2026

Evidence¶

Alexandre Clin Deffarges 2026¶

Alexandre Clin Deffarges, Nataliya Kosmyna, and Pattie Maes. (2026). Socrates went Nuclear: Comparing Interaction Strategies for AI systems in a Learning Context using Brain Sensing. arXiv preprint arXiv:2609.00584. https://arxiv.org/abs/2609.00584 doi:10.48550/arxiv.2609.00584

q2 · i3 · causal · r2

Comparison of attempts to reach the 80% mastery threshold during training, excluding Mode 3 where the adaptive system redirects to sub-questions after incorrect answers. The printed effect size is d = -2.01, described as a "very large effect"; the authors also report that 58% of the exercises of participants in Socratic Mode who abandoned the dialogue mid-way never reached the threshold.

"Socratic participants required nearly twice as many attempts (𝑀= 2.80,𝑆𝐷= 0.73) as Full participants (𝑀= 1.46,𝑆𝐷= 0.59); Welch's𝑡=− 5.865,𝑝<. 0001,𝑑=− 2.01(very large effect)."

Discussion¶

Related Claims¶

• Mastery learning implementation did not reduce material covered or instructional pace, contradicting the Robin Hood contention — the general claim this finding bears on, with the studies behind it
• In eSpark quests requiring three post-quiz attempts, students reached the 80 percent mastery criterion only 14 percent of the time — related
• In an immediate post-test design, participants with an unrestricted chatbot achieved significantly higher learning deltas than participants with a Socratic chatbot (d = 0.80) or an EEG-adaptive exercise system (d = 0.85) — related
• Retrieval practice instructions led students to recall items to a criterion of about three correct retrievals before dropping them — related
• No significant difference in perceived learning across the three modes (p = .845), despite the unrestricted mode's nearly double objective learning delta — related
• Most unrestricted-mode interactions were direct answer retrieval: 52.9% of exercise interactions were copy-paste of the question, and participant answers were linguistically near-identical to chatbot text in 25.3% of cases — related