Automate self-affirmation essay coding with a fine-tuned model while double-coding a random sample for reliability¶

Strategy · All strategies
Evidence · no claims cited

Description¶

The article recommends a structured workflow for scaling coding of open-ended student reflections: train the model on coded essays, validate reliability with sample human coding, apply it to new samples with sample-based validation, and retrain if context shifts. It states: "we strongly recommend that researchers double -code a random sample of the data to ensure reliability when using the fine -tuned model in their work". Automated coding reduces the time and cost of manual coding for large-scale interventions.

Design Implications¶

Context¶

Requirements¶

• A labeled training dataset of coded essays
• Periodic human double-coding of a random sample to verify reliability

Constraints¶

• Researchers must remain vigilant about bias in training data and predictions; the article recommends regular audits of model performance and diverse training data

Target Learners¶

• Middle school students (7th grade) in value-affirmation interventions

Target Learning Goals¶

• Scalable, reliable identification of self-affirming content in student writing to support intervention research and educator feedback

Related Strategies¶

Examples¶

-

Key Sources¶

• Ye, C., Borman, T. H., & Borman, G. D. (2026). Automating Self-Affirmation Essay Coding: Fine-Tuned BERT Performance Comparable to Human Coders and Comparison with GPT-4. Journal of Educational Data Mining, Volume 18, No 1. https://github.com/visortown/bert-self-affirm