Pretraining a multilingual transformer on the large English ASAP dataset and fine-tuning on small target-language data outperforms all monolingual and crosslingual baselines¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? In Study 3, M-BERT models pretrained on about 2000 English answers and fine-tuned on about 270 target-language answers outperformed all three baselines for every target dataset. → Andrea Horbach 2024

Evidence¶

Andrea Horbach 2024¶

Andrea Horbach, Joey Pehlke, Ronja Laarmann-Quante, Yuning Ding. (2024). Crosslingual Content Scoring in Five Languages Using Machine-Translation and Multilingual Transformer Models. Int J Artif Intell Educ. https://doi.org/10.1007/s40593-023-00370-1

q2 · i? · design · r2

Experimental Study 3, a fine-tuning experiment with M-BERT pretrained on the original English ASAP data and fine-tuned per target language via cross-validation. The article reports that "in all cases results outperform all three baselines", e.g. QWK .783 for French versus a .726 monolingual baseline.

"We see that in all cases results outperform all three baselines. The improvement over the monolingual baseline indicates that we indeed benefit from more training data than the approximately 300 training instances in each dataset, even if it is not in the target language."

Discussion¶

Related Claims¶

• Crosslingual content scoring transfer works better between closely related languages, with Chinese the hardest target in this setup — related
• M-BERT with machine translation is the best crosslingual method, outperforming logistic regression with MT and M-BERT zero-shot — related
• Monolingual English BERT shows only a slight advantage over multilingual BERT for English content scoring — related