A response-length-only baseline scores poorly, showing scores depend on broader linguistic features¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2023 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? A Random Forest baseline using only the number of words as predictor performs poorly compared to other models, justifying dependence of scores on other features. → Pakhi Bamdev 2023

Evidence¶

Pakhi Bamdev 2023¶

Pakhi Bamdev, Manraj Singh Grover, Yaman Kumar Singla, Payman Vafaee, Mika Hama, Rajiv Ratn Shah. (2023). Automated Speech Scoring System Under The Lens Evaluating and interpreting the linguistic cues for language proficiency. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-022-00291-5

q2 · i? · design · r2

Baseline comparison in the Results section: a Random Forest regressor and classifier using only response length as predictor were trained alongside the full models. The length-only baseline "performs poorly compared to other models" on both formulations.

"We also observe the Random Forest baseline, which uses only number of words as predictor, performs poorly compared to other models, justifying the dependence of scores on other features introduced previously."

Discussion¶

Related Claims¶

• In the random forest, word count was the most important readability feature while traditional readability formulas ranked near the bottom — related
• Academic word list count, word count, and Flesch-Kincaid grade level are the most important features for predicting cognitive engagement in discussion posts — related
• Adding fluency features to content features sharply increases scoring QWK for every prompt
• Ablation tests identify unique trait-relevant information in specific feature groups: readability and syntactic complexity for content, cohesion and readability for organization, and occurrence, length, or error features for language — related
• Addition tests show trait-specific feature groups that most consistently enhance embedding-based models: morphological complexity for content, length for organization, and lexical sophistication, error, and occurrence features for language — related