Operators׳ adaptation to imperfect automation – Impact of miss-prone alarm systems on attention allocation and performance

• In complex environments alarm systems are not perfectly reliable.
• To maintain system performance operators have to compensate for unreliability.
• We investigate how alarm reliability impacts performance and attention allocation.
• Independent of reliability, performance benefits from working with an alarm system.
• Below a reliability of 70% benefits are associated with increased attentional effort.

Operators in complex environments are often supported by alarm systems that indicate when to shift attention to certain tasks. As alarms are not perfectly reliable, operators have to select appropriate strategies of attention allocation to compensate for unreliability and to maintain overall performance. This study explores how humans adapt to differing alarm reliabilities. Within a multi-task simulation consisting of a monitoring task and two other concurrent tasks, participants were assigned to one of five groups. In the manual control group none of the tasks was supported by an alarm system, whereas the four experimental groups were supported in the monitoring task by a miss-prone alarm system differing in reliability, i.e. 68.75%, 75%, 87.5%, 93.75%. Compared to the manual control group, all experimental groups benefited from the support by alarms, with best performance for the highest reliability condition. However, for the lowest reliability group the benefit was associated with an increased attentional effort, a more demanding attention allocation strategy, and a declined relative performance in a concurrent task. Results are discussed in the context of recent automation research.