A simulated doubling of fake learners (from ~15% to ~26%) significantly increased the bias in Analysis 1A, making 'Doers who are neither Watchers nor Readers' by far the most successful group.¶

Claim · All claims
Evidence · 1 study · associational r1 · q2
Currency · newest study 2019 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? In the 'future projection' simulation duplicating fake-learner data, the effect on Analysis 1A was significant, while the effect on the Tetrad analysis was incremental (same edges, slight weight changes). → Alexandron 2019

Evidence¶

Alexandron 2019¶

Alexandron, G., Yoo, L. Y., Ruipérez-Valiente, J. A., Lee, S., & Pritchard, D. E. (2019). Are MOOC Learning Analytics Results Trustworthy? With Fake Learners, They Might Not Be! International Journal of Artificial Intelligence in Education, 29, 484–506. https://doi.org/10.1007/s40593-019-00183-1

q2 · i? · associational · r1

The 'future projection' simulation duplicated fake-learner data to increase their share from ~15% to ~26%. "The effect on Analysis 1A is significant", with 'Doers who are neither Watchers nor Readers' becoming by far the most successful group, while the Tetrad effect was incremental.

"The effect on the bias of the Tetrad analysis is incremental (same edges, with slight change of weights; see Appendix 2,F i g .9). The effect on Analysis 1A is significant, with "Doers who are neither Watchers nor Readers" becoming by far the most successful group (see Appendix 2,F i g .8)."

Discussion¶

Related Claims¶

• Removing fake learners changed the Tetrad causal graph: the links watching → quizScore and reading → IRT disappeared. — related
• Fake learners solved exercises significantly faster than true learners and showed higher weekly improvement, but did not differ significantly on grade, ability, or CFA.
• Fake learners spent significantly less time on each type of instructional resource than true learners in the MITx 8.MReVx MOOC.
• In the Champaign et al. replication, removing fake learners moved three correlations (Ability vs. Homework Time, Ability vs. Reading Time, Weekly Improvement vs. Homework Time) in the positive direction, but no correlation changed qualitatively.
• Fake learners attempted a smaller proportion of assessment items than true learners, with the smallest gap on homework items.