Validate EDM models across demographic subpopulations of students¶

Strategy · All strategies
Evidence · no claims cited

Description¶

The article recommends that the EDM community routinely use demographic variables to test and validate how well models generalize across diverse populations of students, analogous to the field's earlier shift toward student-level cross-validation. The authors write that "a si milar shift could occur to further validate how well models perform when applied to different subpopulations of students , to ensure that results of EDM analyses are fair and equitable". This requires collecting relevant demographic information during studies, which can be challenging for historical data and sensitive variables.

Design Implications¶

Context¶

Requirements¶

• Relevant demographic information must be collected during studies, since historical data or platforms may lack it

Constraints¶

• Collecting sensitive variables such as race and SES may require additional privacy protections, stricter IRB review, and may reduce participation; it has largely prevented collection of data such as student disability status

Target Learners¶

• All students targeted by EDM-developed models, across demographic subpopulations

Target Learning Goals¶

• Ensuring predictive models of learning outcomes perform fairly and equitably for every student subgroup

Related Strategies¶

• Test publishers and users should routinely investigate longitudinal achievement construct invariance over time
• Mitigating Racial Bias in Edtech Products

Examples¶

-

Key Sources¶

• Paquette, Li, Baker, Ocumpaugh, & Andres. (2020). Who's learning? Using demographics in EDM research. Journal of Educational Data Mining, Volume 12, No 3. https://jedm.educationaldatamining.org/index.php/JEDM/article/view/404