A reliability and usability study of TRACEr-RAV: The technique for the retrospective analysis of cognitive errors – For rail, Australian version

The aim of this study was to compare the usability and reliability of two human error identification tools: TRACEr-Rail (developed by the Rail Safety and Standards Board in the UK) and TRACEr-RAV (an Australian specific version of the tool). Following an attempt to modify TRACEr-Rail to more appropriately suit the Australian rail context, it was predicted that TRACEr-RAV would be rated as more usable and be applied more consistently by Australian users than TRACEr-Rail. In Experiment 1, twenty-five rail employees used either TRACEr-Rail or TRACEr-RAV1 to extract and classify errors from six Australian rail incident reports. In Experiment 2, eleven university students used both TRACEr-Rail and TRACEr-RAV2 to extract and classify errors from three incident summaries. The results revealed that although modification of TRACEr-Rail to become TRACEr-RAV1 and TRACEr-RAV2 did not result in improved inter-rater reliability, modification resulted in improved ratings of usability in Experiment 2. Most participants in Experiment 2 preferred TRACEr-RAV2 to TRACEr-Rail. The poor inter-rater reliability observed was most likely the result of inadequate training, limited practice in using the tools, and insufficient human factors knowledge.