Train AI feedback models on context-specific educational data including student, course and instructor data¶

Strategy · All strategies
Evidence · 4 claims (1 for, 3 mixed) · 1 study (1 causal), q3 · 0 of 1 report an effect size · 4 claims rest on one study

Description¶

The authors recommend moving beyond generic language models by developing AI systems that "can ingest and understand the unique characteristics of a course, students and course instructors." They specify that training data should include student strengths, weaknesses and progress, course content such as syllabus and assessment rubrics, and examples of the instructor's feedback and teaching methods so the AI can emulate the instructor's voice.

Design Implications¶

Context¶

Requirements¶

• Comprehensive context-specific data: student performance and progress data, instructor observations, course content and rubrics, and examples of the instructor's prior feedback

Constraints¶

• The article presents this as a recommendation based on its findings, not a tested intervention

Target Learners¶

• higher education students in programming courses

Target Learning Goals¶

• actionable, context-aligned feedback on student work

Claims¶

• Both instructor and AI-generated feedback were perceived positively, with instructor feedback rated higher descriptively on all four perception dimensions [~W]
• Students receiving instructor feedback showed significantly greater improvements in lab assignment scores than students receiving AI-generated feedback, after controlling for initial knowledge [~W]
• Students perceived instructor feedback as significantly more useful than AI-generated feedback, with no significant differences on fairness, developmental value or encouragement [~W]
• Students' initial knowledge level (midterm scores) did not significantly influence improvement in lab scores, indicating it did not confound the feedback-source difference [+W]

Related Strategies¶

• Use hybrid human–AI feedback models that combine AI's and instructors' complementary strengths
• Next-step post-training strategy: reinforcement learning from human feedback on authentic learning dialogues and scaling student feedback

Examples¶

-

Key Sources¶

• Er, Erkan; Akçapınar, Gökhan; Bayazıt, Alper; Noroozi, Omid; Banihashem, Seyyed Kazem. (2024). Assessing student perceptions and use of instructor versus AI-generated feedback. British Journal of Educational Technology. https://doi.org/10.1111/bjet.13558