Text splicing into single sentences improves ML and BERT Bloom classification on the Scaria OOD dataset¶
Claim · All claims
Evidence · 1 study · designr2·q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? Splitting questions into sentences and taking the maximum Bloom level raised XGBoost to 0.59 and BERT to 0.62 macro F1 on Scaria. → Michael Lawrence Castanares 2026
Evidence¶
Michael Lawrence Castanares 2026¶
Michael Lawrence Castanares, Princess Ventures, and Allan Tan. (2026). Evaluation of Pre-trained Models for Pedagogical Assessment of NovelAI-Assisted Educational Questions. Preprint. https://arxiv.org/abs/2609.27749 doi:10.48550/arxiv.2609.27749
q2 · i? · design · r2
Numerical experiment applying text splicing (splitting a question on punctuation and classifying each sentence) to pre-trained ML and DistilBERT models on the Scaria OOD dataset; improvements printed as "0.59 and 0.62".
"Text splicing improved the performance of ML and BERT in the Scaria dataset (see Table 3). In particular, XGBoost and BERT models F1-scores increased to 0.59 and 0.62, respectively."
Discussion¶
Related Claims¶
- LLMs show robust zero- and few-shot Bloom classification of OOD AI-assisted questions compared with ML and BERT models — related
- Feature-based ML Bloom classifiers trained on IID data degrade sharply on out-of-distribution AI-assisted questions — related
- Model retraining on labeled OOD data provides the largest improvement in Bloom classification across models and datasets — related
- AI-generated OOD question datasets are longer, syntactically deeper, and overlap less with Bloom trigger verbs than human-curated IID data — related
- BERT and MentalBERT remain resilient under sentence shuffling in ENS-based mental health classification
- Model accuracy on Bloom classification drops with question text length, with LLMs degrading least — related
- Appending learning objectives to questions improves Bloom classification on the AF dataset — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
text-splicing-improves-ood-bloom-classification |
title |
Text splicing into single sentences improves ML and BERT Bloom classification on the Scaria OOD dataset |
description |
Text splicing into single sentences improves ML and BERT Bloom classification on the Scaria OOD dataset |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2026 |
recheck_by |
2029 |
generated |
by process:wiki-ingest on 2026-10-10 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — michael-lawrence-castanares-2026 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.