Combine system logs, eye-tracking, and speech recognition to capture professional performance, communication, and collaboration for automated feedback in full-mission simulator training¶

Strategy · All strategies
Evidence · 1 claim (1 mixed) · 1 study (1 qualitative), q2 · 0 of 1 report an effect size · 1 claim rests on one study

Description¶

The article proposes an MMLA design using three technologies: system logs capturing navigational actions such as speed, "angle of turns", and distance between vessels; eye-tracking capturing "area of interest and joint visual attention" as accountable traces of gaze orientation; and speech recognition transcribing bridge team communications, usable in debriefing to provide accountability. Each maps to a display of understanding identified in the episode analyses.

Design Implications¶

Context¶

Requirements¶

• The MMLA system must become well integrated with the simulator system and add value to it

Constraints¶

• Several different simulator manufacturers exist for bridge simulators, and integration with all simulator brands may be difficult

Target Learners¶

• master mariner students in full-mission bridge simulator training

Target Learning Goals¶

• professional navigation performance, communication, and collaboration

Claims¶

• Students raised four categories of concern about potential misuse of MMLA data: judgment, privacy, career impact, and academic integrity [~W]

Related Strategies¶

• Address ethics and privacy considerations — transparency, security, student control, accountability, and informed consent — at the initial stages of MMLA design in simulation-based training
• Triangulate physiological data with other CSCL data streams ( such as video or logfile traces ) to uncover otherwise unidentifiable psychophysiological reactions

Examples¶

-

Key Sources¶

• Charlott Sellberg · Amit Sharma. (2025). Toward multimodal learning analytics in simulation-based collaborative learning: A design ethnography of maritime training. Intern. J. Comput.-Support. Collab. Learn. https://doi.org/10.1007/s11412-024-09435-2