Next-step post-training strategy: reinforcement learning from human feedback on authentic learning dialogues and scaling student feedback¶

Strategy · All strategies
Evidence · no claims cited

Description¶

The article's forward-looking recommendation is to move beyond supervised fine-tuning toward RLHF using authentic learning dialogues, and to incorporate post-trained models into the student journey to quantify student feedback on model performance. The authors state they are "now focused on realizing the full benefit of post-training through reinforcement learning from human feedback (RLHF)," calling it a natural next step given the availability of authentic learning dialogues involving humans.

Design Implications¶

Context¶

Requirements¶

• Availability of authentic learning dialogues involving humans as the basis for RLHF
• More sophisticated evaluations capturing longitudinal interactions between students and tutors

Constraints¶

• The article notes current evaluations relied on automated processes and limited human feedback from non-learners rather than actual students

Target Learners¶

• Students on the Polygence platform who would interact with post-trained tutor models

Target Learning Goals¶

• Improving pedagogical performance of AI tutors beyond what supervised fine-tuning achieves

Related Strategies¶

• Enhance student development agents with finetuned or small task-specific models and richer memory architectures rather than prompting alone
• Planned extensions: expanded user studies, expert-in-the-loop RLHF, hybrid retrieval, and hand-tracking-based assessment
• Train AI feedback models on context-specific educational data including student, course and instructor data

Examples¶

-

Key Sources¶

• Janos Perczel, Jin Chow, and Dorottya Demszky. (2025). TeachLM: Post-Training LLMs for Education Using Authentic Learning Data. https://arxiv.org/abs/2510.05087