BERT was the best unimodal text classifier for Key Stage classification with an F1 score of 0.75¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person · recheck by 2027: tool and model accuracy changes quickly

Subclaims¶

q2 i? Among eight fine-tuned transformers, BERT achieved the highest unimodal F1 score of 0.75. → Jordan J. Bird 2024

Evidence¶

Jordan J. Bird 2024¶

Jordan J. Bird. (2024). What Differentiates Educational Literature? A Multimodal Fusion Approach of Transformers and Computational Linguistics. arXiv preprint. https://arxiv.org/abs/2411.17593 doi:10.48550/arxiv.2411.17593

q2 · i? · design · r2

Results section benchmark of eight transformers (Longformer 148M parameters down to ALBERT 12M), each fine-tuned for 5 epochs on segmented text. BERT scored 0.750 accuracy, 0.751 precision, 0.750 recall and "an F1 score of 0.75" per Table 4.

"The best single modal model overall was BERT, which had an F1 score of 0.75."

Discussion¶

Related Claims¶

• Fusing transformer text classifiers with a linguistic-feature neural network significantly outperforms unimodal models for UK Key Stage classification, with ELECTRA + ANN reaching F1 = 0.996 — related
• Unimodal classification confuses Key Stage 4 with Key Stage 3, while the best multimodal model classifies Key Stages 4 and 5 perfectly — related
• Adding the linguistic neural network to a transformer does not significantly change inference time (p = 0.244) — related
• Neural networks on linguistic features classify Key Stages poorly, with deeper networks performing worse; the best (one hidden layer, 175 ReLU) reached F1 = 0.392 — related