Classifiers systematically confuse adjacent engagement levels: social with active, constructive with interactive, and interactive with active¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2022 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2025: tool and model accuracy changes quickly

Subclaims¶

q2 i? Across all models, approximately 70% of misclassified social posts were labelled active, approximately 55% of misclassified active posts were labelled social, and approximately 40% of misclassified constructive posts were labelled interactive. → Gorgun 2022

Evidence¶

Gorgun 2022¶

Gorgun, G., Yildirim-Erbasli, S. N., & Demmans Epp, C. (2022). Predicting cognitive engagement in online course discussion forums. Proceedings of the 15th International Conference on Educational Data Mining. https://doi.org/10.5281/zenodo.6853149

q2 · i? · design · r2

Error analysis of confusion matrices and misclassified posts across the three classifiers. Similar post length, AWL count, and Flesch-Kincaid grade level of misclassified social and active posts help explain the confusion between these adjacent categories.

"Additionally, the confusion matrix (Table 5) suggested that all classifiers tend to misclassify social engagement with active engagement. Of those misclassified posts, approximately 70% of social posts were classified as active engagement across the predictive models."

Discussion¶

Related Claims¶

• All three trained classifiers outperformed the zero-rule baseline (28.4% accuracy) for classifying cognitive engagement in discussion posts — related
• A support vector machine classifier outperformed decision tree and random forest models in predicting cognitive engagement levels of online discussion posts — related
• The classifier's dominant error was labelling Low Stress students as Moderate Stress, a bias the authors judge safe because over-assignment yields extra support while the dangerous high-as-low error was uncommon — related
• 86% of World of Warcraft forum posts constitute social knowledge construction rather than social banter — related
• Classifier errors cluster into three interpretable patterns, and the unknown abstention class is well classified (F1 = 0.93), reliably filtering off-topic chat — related