A broad linguistic feature set yields 73.2% classification accuracy for three-level proficiency scoring of TOEFL11 essays¶

Claim · All claims
Evidence · 1 study · associational r2 · q2
Currency · newest study 2016 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? A classification model using all 114 linguistic features plus prompt and L1 achieved 73.2% accuracy (10-fold CV) on the balanced TOEFL11SUBSET, against a 33% random baseline. → Sowmya Vajjala 2016

Evidence¶

Sowmya Vajjala 2016¶

Sowmya Vajjala. (2016). Automated assessment of non-native learner essays: Investigating the role of linguistic features. International Journal of Artificial Intelligence in Education. https://arxiv.org/abs/1612.00729

q2 · i? · associational · r2

Classification experiment on TOEFL11SUBSET (1069 texts per proficiency category, 10-fold cross validation with SMO). The all-features model reached "a classification accuracy of 73.2%", with most confusion between adjacent proficiency levels.

"The first classification model with this corpus consisted of all the features described in the Features section. This model achieved a classification accuracy of 73.2%."

Discussion¶

Related Claims¶

• Removing native language (L1) sharply degrades discourse-feature scoring on TOEFL11 but not on FCE — related
• Regression on TOEFL11SUBSET reaches a Pearson correlation of 0.8 with proficiency scores — related
• The most predictive linguistic features differ across datasets, overlapping only in lexical diversity measures — related