Students rate pairs of extracted and generated Wikipedia-based explanations as helpful 83.33% of the time, more than either type alone¶

Claim · All claims
Evidence · 1 study · causal r2 · q2
Currency · newest study 2022 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? When shown one extracted and one generated Wikipedia-based explanation, students rated at least one helpful 83.33% of the time, a difference significant at 95% confidence versus each individual type. → Kochmar 2022

Evidence¶

Kochmar 2022¶

Kochmar, E., Vu, D. D., Belfer, R., Gupta, V., Serban, I. V., & Pineau, J. (2022). Automated Data-Driven Generation of Personalized Pedagogical Interventions in Intelligent Tutoring Systems. International Journal of Artificial Intelligence in Education, 32, 323–349. https://doi.org/10.1007/s40593-021-00267-x

q2 · i? · causal · r2

Second pilot experiment in which students received two randomly-selected explanations after an incorrect solution and picked the most helpful. Extracted explanations were "selected as helpful 55.66% of the time" versus 44.44% for generated ones; the combined-preference 83.33% result was significant at 95% confidence.

"students are far more likely to rate the feedback as overall helpful when both types of explanations are shown to them as compared to only showing extracted explanations."

Discussion¶

Related Claims¶

• Wikipedia-based explanation success rates (17.65%) are far lower than personalized hints (60.47%), with no significant difference between extracted and generated types — related
• A Decision Tree classifier distinguishes high- from low-quality Wikipedia-based explanations with an F1 score of 80.32% on the validation set — related