Toward a real-time model-based training system

This article describes the development of a real-time model-based training system that provides adaptive “over-the-shoulder” (OTS) instructions to trainees as they learn to perform an Anti-Air Warfare Coordinator (AAWC) task. The long-term goal is to develop a system that will provide real-time instructional materials based on learners’ actions, so that eventually the initial set of instructions on a task can be strengthened, complemented, or overridden at different stages of training. The training system is based on the ACT-R architecture, which serves as the theoretical background for the cognitive model that monitors the learning process of the trainee. An experiment was designed to study the impact of OTS instructions on learning. Results showed that while OTS instructions facilitated short-term learning, (a) they took time away from the processing of current information, (b) their effects tended to decay rapidly in initial stages of training, and (c) their effects on training diminished when the OTS instructions were proceduralized in later stages of training. A cognitive model that learned from both the upfront and OTS instructions was created and provided good fits to the learning and performance data collected from human participants. Our results suggest that to fully capture the symbiotic performance between humans and intelligent training systems, it is important to closely monitor the learning process of the trainee so that instructional interventions can be delivered effectively at different stages of training. We proposed that such a flexible system can be developed based on an adaptive cognitive model that provides real-time predictions on learning and performance.