In the Champaign et al. replication, removing fake learners moved three correlations (Ability vs. Homework Time, Ability vs. Reading Time, Weekly Improvement vs. Homework Time) in the positive direction, but no correlation changed qualitatively.¶

Claim · All claims
Evidence · 1 study · associational r2 · q3
Currency · newest study 2019 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? For Ability vs. Homework Time, Ability vs. Reading Time, and Weekly Improvement vs. Homework Time, correlations moved in the positive direction after removing fake learners, with no qualitative change (e.g., no negative correlation became positive). → Alexandron 2019

Evidence¶

Alexandron 2019¶

Alexandron, G., Yoo, L. Y., Ruipérez-Valiente, J. A., Lee, S., & Pritchard, D. E. (2019). Are MOOC Learning Analytics Results Trustworthy? With Fake Learners, They Might Not Be! International Journal of Artificial Intelligence in Education, 29, 484–506. https://doi.org/10.1007/s40593-019-00183-1

q3 · i? · associational · r2

Replication Study 2 replicated Champaign et al.'s correlation analysis on 8.MReVx data. The authors report that "after removing fake learners' data, the correlation moved in the positive direction" for the three significant correlations, with "no qualitative change". Grade vs. Homework/Reading Time showed almost no effect.

"after removing fake learners' data, the correlation moved in the positive direction. However, in each of the correlations we did not see a qualitative change, such as a negative correlation that becomes positive."

Discussion¶

Related Claims¶

• Removing fake learners changed the Tetrad causal graph: the links watching → quizScore and reading → IRT disappeared. — related
• Fake learners spent significantly less time on each type of instructional resource than true learners in the MITx 8.MReVx MOOC. — related
• Fake learners solved exercises significantly faster than true learners and showed higher weekly improvement, but did not differ significantly on grade, ability, or CFA. — related
• Fake learners attempted a smaller proportion of assessment items than true learners, with the smallest gap on homework items. — related
• A simulated doubling of fake learners (from ~15% to ~26%) significantly increased the bias in Analysis 1A, making 'Doers who are neither Watchers nor Readers' by far the most successful group.