Wikipedia-based explanation success rates (17.65%) are far lower than personalized hints (60.47%), with no significant difference between extracted and generated types¶

Claim · All claims
Evidence · 1 study · causal r1 · q2
Currency · newest study 2022 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Success rates after Wikipedia-based explanations were 16.00% (extracted), 16.67% (generated), and 17.65% overall, with no statistically significant difference between the two types. → Kochmar 2022 q2 i? Wikipedia-based explanations produced a much lower success rate than personalized hints. → Kochmar 2022

Evidence¶

Kochmar 2022¶

Kochmar, E., Vu, D. D., Belfer, R., Gupta, V., Serban, I. V., & Pineau, J. (2022). Automated Data-Driven Generation of Personalized Pedagogical Interventions in Intelligent Tutoring Systems. International Journal of Artificial Intelligence in Education, 32, 323–349. https://doi.org/10.1007/s40593-021-00267-x

q2 · i? · causal · r1

Same pilot experiment on the Korbit platform: success was measured by whether the student solved the exercise in their next attempt after receiving explanations. The article reports "no statistically significant difference between the two types" of explanations.

"despite the fact that students find Wikipedia-based explanations helpful, the success rates for such explanations are overall quite low: 17.65% as compared to 60.47% for personalized hints."

Discussion¶

Related Claims¶

• Personalized hints raise student success at solving exercises: 60.47% success before second attempt with the discourse model versus 37.93% baseline (p = 0.03) — related
• Students rate pairs of extracted and generated Wikipedia-based explanations as helpful 83.33% of the time, more than either type alone — related