A guide for assessing control room operator performance using speed and accuracy, perceived workload, situation awareness, and eye tracking

In the petrochemical industry, control room operators must address safety-critical alarms and other tasks using complex interfaces. This study developed a guide for assessing human performance using standard human factors measurement tools, and tested the sensitivity of those tools with two interface designs (i.e., gray and black) at three levels of workload (i.e., easy, medium, and difficult). The guide measures human performance through speed and accuracy, perceived workload using two standard instruments (i.e., NASA Task Load Index (NASA-TLX) and Subjective Workload Assessment Technique (SWAT)), situation awareness through the Situation Awareness Global Assessment Technique (SAGAT), and gaze through eye tracking coordinates. Twelve engineering student participants completed one simulation session at each of the three workload levels using one of two interface designs. Workload was manipulated through the number of simulated events (failures) in each session. Overall, the speed and accuracy measures, workload ratings, and eye tracking showed sensitivity to differences in workload level, and situation awareness showed sensitivity to the interaction between workload level and interface type. None of the tools were sensitive to interface type alone. Accuracy was highest under easy workload. Time per failure decreased at higher workload levels. Perceived workload ratings from the SWAT increased as workload increased, but workload ratings from the NASA-TLX were not different across workload levels. When workload increased, situation awareness remained steady for the gray interface but decreased sharply for the black interface, illustrating an interaction effect. Finally, the percentage of time spent looking at different areas of the screen during steady-state periods differed among workload levels. The tools in this guide can be used in the petrochemical industry to make design decisions for control room interfaces when workload levels are a concern.