Coding ATC incident data using HFACS: Inter-coder consensus

Reliability studies for coding contributing factors of incident reports in high hazard industries are rarely conducted and reported. Although the Human Factors Analysis and Classification System (HFACS) appears to have a larger number of such studies completed than most other systems doubt exists as the accuracy and comparability of results between studies due to aspects of methodology and reporting. This paper reports on a trial conducted on HFACS to determine its reliability in the context of military air traffic control (ATC). Two groups participated in the trial: one group comprised of specialists in the field of human factors and the other group comprised air traffic controllers. All participants were given standardised training via a self-paced workbook and then read 14 incident reports and coded the associated findings. The results show similarly low consensus for both groups of participants. Several reasons for the results are proposed associated with the HFACS model, the context within which incident reporting occurs in real organisations and the conduct of the studies.

► A reliability study was conducted on HFACS in the context of military air traffic control.
► The study tested inter-coder consensus in a real-world, small organisation environment.
► Two groups were tested on coding consensus: air traffic controllers and human factors specialists.
► Low consensus was found at the category level of HFACS for both groups of participants.
► Doubt exists as to the reliability of HFACS taxonomic coding in real world small organisations.