Feature-based short-answer scoring systems may be less susceptible to certain student gaming attempts than end-to-end neural models¶

Claim · All claims
Evidence · 1 study · review r2 · q1
Currency · newest study 2023 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q1 i? Ding et al. (2020)'s adversarial experiments suggest classical feature-based systems might be less susceptible to certain types of gaming and cheating attempts by students than end-to-end neural models. → Xiaoyu Bai · Manfred Stede 2023

Evidence¶

Xiaoyu Bai · Manfred Stede 2023¶

Xiaoyu Bai · Manfred Stede. (2023). A Survey of Current Machine Learning Approaches to Student Free-Text Evaluation for Intelligent Tutoring. International Journal of Artificial Intelligence in Education (2023) 33:994–1032. https://doi.org/10.1007/s40593-022-00323-0

q1 · i? · review · r2

The survey reports Ding et al. (2020)'s adversarial-setting experiments on short-answer scoring, highlighting the vulnerability of both feature-based and neural ASAS models to adversarial input imitating gaming attempts. The finding is hedged as a suggestion; no effect size is printed.

"Ding et al. ( 2020)'s experiments on short-answer scoring in an adversarial setting suggest that classical, feature-based systems might be less susceptible to certain types of gaming and cheating attempts by students than end-to-end neural models are"

Discussion¶

Related Claims¶

• Single composite accountability indicators are susceptible to gaming and Campbell's Law corruption pressures — related
• Concatenating hand-crafted linguistic features with neural essay representations significantly increases essay scoring performance — related
• Where labelled training data is limited, adding hand-crafted features improves the performance of neural text evaluation models — related