NCA scores of IIU input vectors order action types identically to the main-effects ordering (Spearman's ρ = 1)¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person · recheck by 2027: tool and model accuracy changes quickly

Subclaims¶

q2 i? The ordering of action types by mean NCA scores of IIU vectors perfectly matches the ordering by average probability of correct responses, with Spearman's rank correlation ρ = 1. → Lu 2024

Evidence¶

Lu 2024¶

Lu, Y., Tong, L., & Cheng, Y. (2024). Advanced Knowledge Tracing: Incorporating Process Data and Curricula Information via an Attention-Based Framework for Accuracy and Interpretability. Journal of Educational Data Mining, 16(2). https://osf.io/mdpzc/ doi:10.31219/osf.io/3k7z2

q2 · i? · no effect size could be confirmed in the full text · design · r2

Analysis of real action logs: IIU vectors were saved, resampled with SMOTE (reference class "answer requested", count 686862), and transformed via Neighborhood Components Analysis (Table 7). The authors report "a perfect Spearman’s rank correlation (ρ = 1)".

"This alignment is further substantiated by a perfect Spearman’s rank correlation (ρ = 1), indicating a direct correspondence between the two orders."

Discussion¶

Related Claims¶

• In artificial action logs, \"answer requested\" yields the lowest average predicted probability of a correct end-of-unit response, below even \"wrong response\" — related
• Single instances of help-seeking actions (answer requested, explanation requested) carry more predictive information than single correct or open responses (ISA) — related
• Component-level embeddings make CourseGraph interpretable: learning outcomes contributed most to an example overlap decision — related
• A wrong response preceding a help request is associated with a higher predicted probability of a correct end-of-unit response than the request alone — related