Within-user exercise accuracy does not differ significantly between high- and low-score sessions¶

Claim · All claims
Evidence · 1 study · causal r2 · q3
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? Within-user exercise accuracy differences between above-average and below-average score sessions were not statistically significant (0.76 vs. 0.70, N=56, p=0.281). → Po-Chin Chang 2026

Evidence¶

Po-Chin Chang 2026¶

Po-Chin Chang, Nicholas Hogan, Aske Plaat, Michiel T. van der Meer. (2026). Learning to Prompt: Improving Student Engagement with Adaptive LLM-based High-School Tutoring. https://arxiv.org/abs/2606.20138 doi:10.48550/arxiv.2606.20138

q3 · i? · causal · r2

Within-user analysis from the live A/B deployment comparing same-student sessions where the AI scored above versus below its average. Accuracy of 0.76 vs. 0.70 was "not statistically significant" (p=0.281); equivalence was not tested.

"Furthermore, the subsequent within-user exercise accuracy differences were not statistically significant (0.76 vs.0.70,N= 56,p= 0.281, App. P)."

Discussion¶

Related Claims¶

• High- and low-performing students did not differ significantly on the midterm-exam ENA network dimensions despite detected network differences — related
• Across three physics assessments, students who declared GenAI use scored slightly higher than non-users, but none of the differences were statistically significant — related