BERT outperforms other models once trained on about 20 to 30% of the training data, with all performance curves flattening as training size increases¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2023 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? BERT with non-random word-order performs lower than FFNN and LSTM for smaller training data sizes but outperforms all other models at about 20 to 30% of the training data. → Wulff 2023

Evidence¶

Wulff 2023¶

Wulff, P., Mientus, L., Nowak, A., & Borowski, A. (2023). Utilizing a Pretrained Language Model (BERT) to Classify Preservice Physics Teachers' Written Reflections. International Journal of Artificial Intelligence in Education, 33, 439–466. https://doi.org/10.1007/s40593-022-00290-6

q2 · i? · design · r2

Sensitivity analysis varied training data proportions from 0.05 to 1.00 with single random draws, assessing performance on the test dataset using the most performant RQ1 models. Results shown in Fig. 2 indicate BERT "outperforms all other models at about 20 to 30%" of training data.

"Figure 2 indicates that BERT with non-random word-order performs lower for smaller training data sizes compared to FFNN and LSTM, but outperforms all other models at about 20 to 30% of the size of the entire training data."

Discussion¶

Related Claims¶

• Fine-tuned BERT outperforms FFNN and LSTM deep learning architectures for classifying segments of written reflections, with weighted F1 average of .82 — related