Augmentation levels below 20x produce unreliable classification performance in small educational datasets¶

Claim · All claims
Evidence · 1 study · causal r2 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Classification performance was generally unreliable at augmentation levels below 20x for both the French and English datasets across the four BERT models. → Keith Cochran 2025

Evidence¶

Keith Cochran 2025¶

Keith Cochran, Clayton Cohn, Jean-François Rouet, Peter Hastings. (2025). Comparing Text Augmentation by GPT-3.5 and Llama3 for Evaluating Student Responses. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-025-00473-x

q2 · i? · causal · r2

Performance curves across augmentation levels 0x–100x (Fig. 2 for French; Figs. 3–6 for English rainwater concepts) showed that "augmentation levels below 20x resulted in unreliable classifications in the majority of cases".

"At levels of augmentation less than 20x, classification performance with all of the BERT models was generally unreliable."

Discussion¶

Related Claims¶

• LLM-generated responses outperform self-augmentation when augmenting small educational classification datasets — related
• Fine-tuning BERT-based classifiers on LLM-augmented student-response data outperforms unaugmented training for all tested concepts — related
• Datasets with entropy below 0.5 are difficult to augment enough to beat majority-label guessing — related
• Self- and GPT-augmented performance peaks and degrades after 55x–89x augmentation, while Llama-augmented performance keeps rising toward 100x — related