Concatenating hand-crafted linguistic features with neural essay representations significantly increases essay scoring performance¶

Claim · All claims
Evidence · 1 study · review r2 · q2
Currency · newest study 2023 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Uto et al. (2020) observed significant performance increases in essay scoring when hand-crafted features were concatenated with neural representations at the pre-output layer. → Xiaoyu Bai · Manfred Stede 2023

Evidence¶

Xiaoyu Bai · Manfred Stede 2023¶

Xiaoyu Bai · Manfred Stede. (2023). A Survey of Current Machine Learning Approaches to Student Free-Text Evaluation for Intelligent Tutoring. International Journal of Artificial Intelligence in Education (2023) 33:994–1032. https://doi.org/10.1007/s40593-022-00323-0

q2 · i? · review · r2

The survey reports Uto et al. (2020)'s hybrid essay scoring experiments, in which neural representations from BERT or LSTMs were concatenated with manual feature vectors; "they observed significant performance increases" when features were added. No effect size is printed.

"When hand-crafted features were added, they observed significant performance increases."

Discussion¶

Related Claims¶

• Feature-based short-answer scoring systems may be less susceptible to certain student gaming attempts than end-to-end neural models — related
• Holistic scoring of German university students' essays posed significant challenges to both a neural model and an SVM with n-gram features — related
• Where labelled training data is limited, adding hand-crafted features improves the performance of neural text evaluation models — related
• Fusing transformer text classifiers with a linguistic-feature neural network significantly outperforms unimodal models for UK Key Stage classification, with ELECTRA + ANN reaching F1 = 0.996 — a broader claim this one bears on