On the reliability and validity of ship-ship collision risk analysis in light of different perspectives on risk

A number of authors have discussed reliability and validity of quantitative risk analysis (QRA). These concepts address respectively whether a QRA provides the same risk picture when the analysis is repeated and whether the analysis addresses the right concept. While it has been argued that QRA is not in general reliable, there is little evidence supporting this claim available in the scientific literature. In light of this, this paper studies the reliability of QRA through a case study of ship-ship collision risk. It is found that probability- and indicator based risk perspectives do not necessarily provide a reliable risk picture, neither in terms of numerical accuracy of the risk metrics, nor in terms of rank order of risk metrics in various parts of the system. The results of the case study indicate a low inter-methodological reliability for the selected methods, raising concerns about their validity. This is discussed applying criteria concerning validity of risk analysis and in terms of the validity of the proposed encounter detection mechanisms. Significant uncertainty is found regarding this encounter definition in the selected methods, implying a need for more focus on this important aspect of maritime traffic risk analysis.