The most predictive linguistic features differ across datasets, overlapping only in lexical diversity measures¶

Claim · All claims
Evidence · 1 study (2 entries) · associational r2 · q2
Currency · newest study 2016 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? ReliefF-selected top features on TOEFL11SUBSET and FCE overlap only on TTR and corrected TTR; error and discourse-chain features swap roles between corpora. → Sowmya Vajjala 2016

Evidence¶

Sowmya Vajjala 2016¶

Sowmya Vajjala. (2016). Automated assessment of non-native learner essays: Investigating the role of linguistic features. International Journal of Artificial Intelligence in Education. https://arxiv.org/abs/1612.00729

q2 · i? · associational · r2

Comparison of ReliefF top-10 feature lists across the two corpora. The two datasets share "only with respect to the lexical diversity features (TTR and CTTR)"; co-reference chain features were best for FCE but not TOEFL11.

"It is interesting to note that compared to the best features on TOEFL11S UBSET , there is an overlap only with respect to the lexical diversity features (TTR and CTTR)."

Sowmya Vajjala 2016 (2)¶

Sowmya Vajjala. (2016). Automated assessment of non-native learner essays: Investigating the role of linguistic features. International Journal of Artificial Intelligence in Education. https://arxiv.org/abs/1612.00729

q2 · i? · associational · r2

Error features also diverge: percentage of spelling errors was most predictive for TOEFL11SUBSET, while non-spelling and all errors were most predictive for FCE.

"With respect to the error features, while percentage of spelling errors was the most predictive feature or TOEFL11S UBSET , non-spelling errors and all errors were the most predictive for FCE."

Discussion¶

Related Claims¶

• Type-token ratio showed a non-monotonic relationship with predicted difficulty, with both very low and very high lexical diversity increasing predicted text difficulty — related
• Feature operationalization can reverse the direction of the relationship with essay score, as with document length and TTR variants — related
• SHAP analysis identifies Hapax Ratio as the most influential stylometric feature in Random Forest predictions, followed by NSR, Noun Ratio, Adverb Ratio, and TTR — related
• A broad linguistic feature set yields 73.2% classification accuracy for three-level proficiency scoring of TOEFL11 essays — related
• Removing native language (L1) sharply degrades discourse-feature scoring on TOEFL11 but not on FCE — related
• Systematic linguistic differences exist between demographic groups in PISA text responses, including response length, lexical diversity, and out-of-vocabulary tokens — related
• Addition tests show trait-specific feature groups that most consistently enhance embedding-based models: morphological complexity for content, length for organization, and lexical sophistication, error, and occurrence features for language — related