Question specificity during Socratic AI tutoring correlates mildly but positively with self-reported expected course grade (Pearson r ≈ 0.43), with substantial scatter¶

Claim · All claims
Evidence · 1 study · associational r1 · q2 · i3 large
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i3 The fraction of a student's questions classified as specific correlated positively with expected course grade (Pearson r ≈ 0.43, p < 0.0001), but the relationship was far from perfectly predictive. → Syed Furqan Abbas Hashmi 2025

Evidence¶

Syed Furqan Abbas Hashmi 2025¶

Syed Furqan Abbas Hashmi, N. Sanjay Rebello. (2025). Analyzing Undergraduate Problem-Solving in Physics Through Interaction With an AI Chatbot. https://arxiv.org/abs/2508.14778 doi:10.48550/arxiv.2508.14778

q2 · i3 · associational · r1

Correlational analysis linking each student's question specificity (fraction of questions coded specific) to self-reported expected course grade in Week 9. The article reports "Pearson r ≈ 0.43, p < 0.0001" and notes considerable scatter, with some high-specificity students expecting only mid-range grades.

"A mild positive correlation was observed (Pearson r ≈ 0.43, p < 0.0001). In general, students who engaged in a higher proportion of specific, detailed questioning during the chatbot session reported higher expected grades."

Discussion¶

Related Claims¶

• Survey scores show directional consistency with dialogue behavior: students with higher knowledge-based survey scores tended to have above-average question specificity and higher expected grades — related
• Students' questions shift from broad to specific during Socratic AI tutoring, from about 10–15% specific in the first turn to all students asking specific questions by the final turn — related
• mPFC activity during schema-related encoding predicts normalized second-year course grades, with no such relation for MTL activity — related