Combine log-derived behavioral engagement measures with process data and self-report to assess learner engagement in AI-supported practice¶

Strategy · All strategies
Evidence · no claims cited

Description¶

The article's future-directions section recommends extending its observed practice effort approach: "Future work could combine such measures with process data (e.g., timing, revision behavior) or self-report measures to provide a more comprehensive account of learner engagement." The strategy pairs rubric-scored conversation-log submissions with timing, revision, and self-report data so engagement measurement captures both behavior and motivational states.

Design Implications¶

Context¶

Requirements¶

• Submitted AI-agent conversation logs and a consensus-developed scoring rubric as the baseline behavioral measure

Constraints¶

• The article notes log-derived effort "reflects behavioral indicators rather than internal motivational states"

Target Learners¶

• undergraduate students interacting with AI agents during practice sessions

Target Learning Goals¶

• measuring behavioral engagement during AI-supported learning activities

Related Strategies¶

• Monitoring Progress
• Design TTXs with assessment in mind so that log data capture the learning process
• Use multimodal data and time-sensitive analyses to capture situational motivational processes

Examples¶

-

Key Sources¶

• Mahir Akgun and Sacip Toker. (2026). Do Gains from Generative AI–Enabled Adaptive Pretesting Persist? Evidence from a Retention Study. https://arxiv.org/abs/2606.22328