In the Overview dimension, augmentation produced a marginal QWK decrease (ΔQWK = −0.006) despite improving macro-F1, the only dimension with this trade-off¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Augmentation lowered Overview QWK from its baseline while raising macro-F1 from 0.325 to 0.352, the only such dimension. → Firdausi 2026

Evidence¶

Firdausi 2026¶

Firdausi, H.; Wasis; Sholikah, R. W.; Lemantara, J.; Rosyadi, F. D.; Ginardi, R. V. H.; Pradityo, M. H. A.; Hakim, A. R. (2026). Educational Innovation through Automated Essay Scoring: A Multidimensional Framework for Evaluating Critical Thinking in High School Physics Essays. International Journal of Intelligent Engineering and Systems, Vol.19, No.8. https://doi.org/10.22266/ijies2026.0831.09

q2 · i? · design · r2

Ablation result for the Overview dimension (Results §4.3.6, best model IB-BiLSTM-F, test QWK 0.474). The article reports the marginal QWK decrease alongside the macro-F1 improvement and attributes it to synthetic samples possibly blurring ordinal transitions.

"data augmentation produced a marginal decrease in QWK (ΔQWK = −0.006), despite improving macro-F1 from 0.325 to 0.352, the only dimension where this trade -off occurred."

Discussion¶

Related Claims¶

• Data augmentation was critical for ordinal discriminability, with QWK dropping to near-zero without it in the Focus dimension and gains up to +0.346 in severely imbalanced dimensions — related
• The best AES models per FRISCO dimension reached strong agreement for Situation (QWK 0.728) and Clarity (QWK 0.763) but only fair agreement for Focus, Reason, and Inference in Indonesian physics essays — related
• CoTAL improves GPT-4 scoring agreement with human scorers on the Rules Task, raising average subscore QWK from 0.826 to 0.916 (a 10.9% gain) over a non-prompt-engineered baseline — related
• Generative AI platforms cannot yet independently lead meaningful innovation; at best they augment human intelligence for innovation — related
• On the Debugging Task, CoTAL raised average subscore QWK from 0.567 to 0.706 (24.5% improvement) and Total Score QWK by 0.218 (a 38.9% gain) over the Baseline — related