Gains from automated scoring concentrated in linguistic accuracy and logical coherence but were absent in information fidelity and delivery fluency¶

Claim · All claims
Evidence · 1 study · causal r2 · q2 · i3 large
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i3 The time × group interaction was significant for linguistic accuracy (η2ₚ = 0.231) and logical coherence (η2ₚ = 0.109) but not for information fidelity or delivery fluency. → Chen C and Liu T 2026

Evidence¶

Chen C and Liu T 2026¶

Chen C and Liu T (2026) Impact of automated scoring on interpreting performance and self-regulated learning: evidence from a pedagogical experiment. Front. Psychol. 17:1867463. https://doi.org/10.3389/fpsyg.2026.1867463

q2 · i3 · causal · r2

Four dimension-level ANOVAs on pre/post sub-scores found significant interactions with printed effect sizes "η2ₚ = 0.231" (linguistic accuracy) and "η2ₚ = 0.109" (logical coherence); the experimental group gained 3.21 points on accuracy and 2.08 on coherence, while fidelity and fluency "barely moved in the experimental group (+0.04 and +0.18)".

"The time × group interaction was significant for linguistic accuracy [F(1, 44) = 13.20, p < 0.001, η2ₚ = 0.231] and logical coherence [F(1, 44) = 5.40, p = 0.025, η2ₚ = 0.109] but not for information fidelity [F(1, 44) = 1.42, p = 0.240, η2ₚ = 0.031] or delivery fluency [ F(1, 44) = 0.099, p = 0.755, η2ₚ = 0.002]"

Discussion¶

Learner Variables¶

• Writing Skills — outcome: instruction changes it

Related Claims¶

• The automated scoring system's agreement with human raters varied widely by dimension, with information fidelity showing a weak, non-significant correlation — related
• Empirical gains from AI tools in Arab EL2 classrooms are most consistent for surface-level outcomes, while higher-order writing quality and speaking proficiency show mixed results contingent on teacher mediation — a broader claim this one bears on
• AI-peer integrated feedback yields significant gains on all four writing dimensions, with the group × dimension interaction showing differential improvement favoring task achievement and lexical resource over coherence and cohesion — related
• Automated scoring practice produced a significant time × group interaction on total interpreting scores — a broader claim this one bears on
• Gains concentrated in proposed-question scores rather than diagnosis scores, with the largest improvement on Hazard C diagnosis — a narrower finding that bears on this claim
• Higher education students' interactions with an AI automated feedback tool can support the development of their feedback literacy capabilities — related
• Current AI instructional-video systems score higher on content dimensions (Accuracy, Logic) than on delivery dimensions (Engagement, Adaptability), with Adaptability lowest — related
• Student interpreters who practiced with an automated scoring system outperformed the control group on total interpreting scores at post-test — a broader claim this one bears on