Experts, Bayesian Belief Networks, rare events and aviation risk estimates

Bayesian Belief Networks (BBN) are conceptually sensible models for aviation risk assessment. The aim here is to examine the ability of BBN-based techniques to make accurate aviation risk predictions. BBNs consist of a framework of causal factors linked by conditional probabilities. BBN conditional probabilities are elicited from aviation experts. The issue is that experts are not being asked about their expertise but about others’ failure rates. A simple model of expertise, which incorporates the main features proposed by researchers, implies that a best-expert’s estimates of failure rates are based on accessible quantitative data on accidents, incidents, etc. Best-expert estimates will use the best available and accessible data. Depending on the frequency of occurrence, this will be data on similar events, on similar types of event, or general mental rules about event frequencies. These considerations, plus the need to be cautious about statistical fluctuations, limit the accuracy of conditional probability estimates. The BBN framework assumes what is known as the Causal Markov Condition. In the present context, this assumes that there are no hidden common causes for sequences of failure events. Examples are given from safety regulation comparisons and serious accident investigations to indicate that common causes may be frequent occurrences in aviation. This is because some States/airlines have safety cultures that do not meet ‘best practice’. BBN accuracy might be improved by using data from controlled experiments. Aviation risk assessment is now very difficult, so further work on resilience engineering could be a better way of achieving safety improvements.

► Bayesian Belief Networks are conceptually sensible aviation risk models.
► BBNs use elicited failure rate conditional probabilities from aviation experts.
► Simple models show expert’s estimates use accessible data on accidents, incidents.
► Added issues are statistical fluctuations and common causes for failure sequences.
► These together place inherent limits on the accuracy of risk estimates.