Sampling strategies in density-based sensitivity analysis

Decision and policy-makers benefit from the utilization of computer codes in an increasing number of areas and applications. Several authorities and agencies recommend the utilization of proper sensitivity analysis methods in order to confidently entrust model results. In this respect, density-based techniques have recently attracted interest among academicians and practitioners, for their property to characterize uncertainty in terms of the entire distribution of an output variable. However, their estimation is a challenging task and, without a proper methodical approach, errors in the estimates can lead to misleading conclusions. In this work, we propose sampling plans for reducing the computational burden of sensitivity estimates while improving and controlling the accuracy in the estimation. We compare designs based on column substitutions and designs based on permutations. We investigate their behaviour in terms of type I and type II errors. We apply the methods to the Level E model, a computational tool developed by the Nuclear Energy Agency of the OECD for the assessment of nuclear waste disposal sites. Results show that application of the proposed sampling plans allows one to obtain confidence in the sensitivity estimates at a number of model runs several orders of magnitude lower than a brute-force approach. This assessment, based upon the entire distribution of the model output, provides us with ways to effectively reduce uncertainty in the model output, either by prioritizing the model factors that need to be better known or by prioritizing the areas where additional modelling efforts are needed.