An applicability index for reliable and applicable decision trees in water quality modelling

Data-driven environmental models are mainly assessed on the basis of their model fit and only limited attention is given to their applicability for end-users. In this paper, we present the applicability index (API) that scores decision trees in terms of their interpretability and applicability for end-users. The API integrates two criteria, viz. the simplicity of the model and its ability to predict the classes of the response variable. We developed 10,000 decision trees with different parameterizations and assessed the use of API for model selection with two different datasets. The API reduced the number of decision trees that were retained only based on statistical criteria from 2,806 to 173 and from 1,117 to 784, respectively. The models that were retained were more easily interpretable, equally statistically reliable but less complex. Conventional statistical criteria such as Cohen's kappa and the number of correctly classified instances were only moderately correlated with the API (r = 0.26 and r = 0.49, respectively). This indicates that the API is a useful complement to the existing statistical criteria available for model selection. The API was tested for two datasets consisting of water quality data in lowland rivers in Belgium and the Netherlands, hence its validity needs to be tested for other types of data and modelling domains.