All three trained classifiers outperformed the zero-rule baseline (28.4% accuracy) for classifying cognitive engagement in discussion posts¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2022 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2025: tool and model accuracy changes quickly

Subclaims¶

q2 i? Decision tree (60%), random forest (66%), and SVM (71%) full-model accuracies all exceeded the 28.4% zero-rule baseline, and Cochran's Q showed significant differences between classifiers. → Gorgun 2022

Evidence¶

Gorgun 2022¶

Gorgun, G., Yildirim-Erbasli, S. N., & Demmans Epp, C. (2022). Predicting cognitive engagement in online course discussion forums. Proceedings of the 15th International Conference on Educational Data Mining. https://doi.org/10.5281/zenodo.6853149

q2 · i? · design · r2

Results section comparison of the three classifiers against the zero-rule baseline on the 30% test split, with Cochran's Q (Q = 55.68, p < .001) comparing classification accuracies across models. All models "outperformed this simple baseline".

"Note that the accuracy of the zero-rule classifier for the full model would be 28.4%. As can be seen in Table 4, all of the models outperformed this simple baseline. Moreover, Cochran's test revealed statistically significant differences between the classifiers we built, Q = 55.68, p < .001."

Discussion¶

Related Claims¶

• Classifiers systematically confuse adjacent engagement levels: social with active, constructive with interactive, and interactive with active — related
• Academic word list count, word count, and Flesch-Kincaid grade level are the most important features for predicting cognitive engagement in discussion posts — related
• A support vector machine classifier outperformed decision tree and random forest models in predicting cognitive engagement levels of online discussion posts — related
• Discipline-general academic vocabulary (AWL use) supports cognitive engagement identification and may aid generalization across courses
• A random forest classified flagged versus non-flagged word problems with the best accuracy of five tested models (AUC = 0.75) — related
• Random Forest is the most commonly used NLP-supporting algorithm in LA, while deep learning models (LSTM, BERT) have been widely adopted since 2021 — related
• Random forest algorithms yielded the best classification performance in K–8 MMLA studies comparing multiple machine learning models — related
• Algorithmically inferred measurement functions may intensify, rather than resolve, the problem of nomic measurement in analyzing student learning — related
• An AST-only structural classifier substantially exceeds random baseline accuracy in predicting CogTax levels, with L1–L3 mutually confused and L4 perfectly precise — related
• Decision trees achieve competitive accuracy with black-box models (LR, RF, SVM), which performed slightly better but lack interpretability — related
• An LSTM-based strategy temporal analytics framework outperforms random forest, SVM, and majority-class baselines for stealth assessment predictive accuracy — related
• Decision tree classifiers trained on slide interaction features predict end-of-module outcomes with high accuracy across four K-12 programming courses — related