LDA topics capture unique information not covered by Likert-scale SET questions¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2020 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? The R2 values of the multilevel models predicting topic probabilities were low (1.0%-14.6% in the 2017-18 sample), indicating that the Likert scales explained only a small proportion of the variance in topic probabilities and that the LDA topics provide unique information not captured by the closed-ended questions. → Hujala 2020

Evidence¶

Hujala 2020¶

Hujala, M., Knutas, A., Hynninen, T., & Arminen, H. (2020). Improving the quality of teaching by utilising written student feedback: A streamlined process. Computers & Education. https://doi.org/10.1016/j.compedu.2020.103965

q2 · i? · design · r2

Statistical evaluation of the 2017-18 sample using multilevel regression, reporting Level 1 (student level) explained variance for each topic model. The authors conclude that "it seems that the LDA topics also can provide unique information not covered by the Likert-scale questions".

"However, the R2 values of the models predicting topic probabilities are relatively low (1.0% - 14.6%), indicating that the Likert scales can explain, at best, only 14.6% (see Topic 6 in Table 5) of the variances in topic probabilities."

Discussion¶

Related Claims¶

• LDA topic modelling of 6,087 written student feedback responses yields six SET-related topics — related
• Likert-scale SET variables predict topic probabilities for most topics in both academic years — related
• Topic 4 (dissatisfaction with workload) contradicts thematic analysis and appears invalid — related
• Feedback questionnaire wording affects LDA topic results, more for emotional value than content — related