Students identified lack of sensitivity to linguistic nuance as a limitation of AWA feedback¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2016 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Students reported that AWA failed to detect summary sentences signalled by adverbs like 'thus' and 'evidently', missed introductory remarks, and sometimes deemed mere descriptions important while missing substantive argumentative content. → Knight 2016

Evidence¶

Knight 2016¶

Knight, S., Buckingham Shum, S., Ryan, P., Sándor, Á., & Wang, X. (2016). Designing Academic Writing Analytics for Civil Law Student Self-Assessment. International Journal of Artificial Intelligence in Education, 28(1), 1–28. doi:10.1007/s40593-016-0121-0

q2 · i? · design · r2

Theme from qualitative coding of student survey comments, written reflections and lecturer notes in NVivo. One student noted the tool detected phrases like 'in conclusion' but that "the use of adverbs such as 'thus,' and 'evidently,' in conclusive statements failed to be recognized". Another worried that relying on signposts constricts writing style.

"Another limitation is that certain sentences, which were recounts or mere descriptions were deemed important, whilst more substantive parts of the essay containing arguments and original voice failed to be detected."

Discussion¶

Learner Variables¶

• Syntax — moderator: an instructional effect differs with it

Related Claims¶

• Students reported AWA highlighting provoked reflection on their writing despite questioning its accuracy — a broader claim this one bears on
• Students perceived peer group feedback as contextually rich but sometimes subjective, and ChatGPT-4o feedback as more objective and well structured but occasionally lacking contextual sensitivity — related
• ChatGPT-3.5 feedback on EAP argumentative essays aligns closely with human instructor scoring and identifies surface-level issues, but is less effective on rhetorical structure and contextual nuance — related
• Business Writing students valued AI feedback for higher-order and sentence-level comments but found it sometimes too general, while appreciating peers' contextual knowledge — related
• LLM-generated feedback is more comprehensive in volume than human feedback but is sometimes too generic, abstract, or misaligned with the assigned grade — related
• Students showed clear appetite for automated writing support and comfort with machine-generated feedback — related